InfoWave

Your text

(Current Sentence)

There is an open call from the ESRC to use existing datasets to address new problems. There will be a bid submitted in March by Rachael, Christine, and Joel. The bid will aim to fund a corpus-based project which will provide crucial insights into the adaptive function of information density in language. The project will shed new light on how information density is adaptively optimised in language. It will also aim to create tools that can be used by researchers in cognitive science, diagnosticians assessing language impairments, and as a general aid for use in improving writing clarity. These tools will be based primarily on word probabilities (normalised frequencies) from large samples of natural language. For use by researchers, code will be made available for analysing information density in text. For general public use, this code will be integrated with an easy to use web-based analytical tool which can show the current informtion density of a text, and suggest alternative optima.

Alternative Sentence: In March, Christine will be submittting a bid with Joel and Rachael

Optimal DORM: (Current DORM: )

DORM trajectory of entire text

New Text

Paste text here

About

What is this tool for?

This tool currently under development, so this is just a demo.

The tool will be for analysing the information density of text. The tool can be used by researchers in cognitive science, practitioners in health and business communication, diagnosticians assessing language impairments, or by anyone who might find information density analysis useful. This tool uses word probabilities (frequencies) from large samples of natural language to estimate the trajectory of information in a given sentence. Users will be able to find out what the information density of their text is, and the tool will suggest alternatives for each sentence with more optimised information density (where a more optimal distribution is possible).

Ideally, information should be distributed throughout a sentence evenly, rather than clustering at a certain point. This even, uniform distribution means that anything that interrupts a hearer or reader's processing of the sentence - from noise to short lapses in attention - is less likely to impair their overall understanding of the utterance or text. We feed the word probabilities in any sentence you provide to an algorithm which attempts to distribute information more uniformly if it can. The tool will show you the actual information density of your text, as well as a potential optimum.

Click 'Analyse' on the left to see what the tool will look like.

What

What is this all about?

This tool doesn't exist yet, but this is meant to show what it will look like once development has been completed. Read more about the tool and the project that will develop it below.

Who

Who made this tool?

The tool will be developed as part of the CAIL Project.

How

How does it work?

This tool uses word probabilities (normalised frequencies) from large samples of natural language to estimate the trajectory of information in a given sentence. According to information theory, lower probability (frequency) words are generally more informative. Ideally, information should be distributed throughout a sentence evenly: this means that anything that interrupts a reader or hearer's processing of the sentence - from noise to short lapses in attention - is less likely to impair their overall understanding of the utterance or text. We feed the word probabilities in any sentence you provide to an algorithm which attempts to distribute information more uniformly if it can. The tool will show you the actual information density of your text, as well as a potential optimum. The tool uses both raw information content based on single word probabilities, as well as calculating a DORM (deviation of rolling means) for the information content across a sentence.

Get

Can I use this tool for my work?

The tools developed as part of the CAIL project are freely available. Code for all the tools will be available on Github. These materials will be free to use and adapt under a CC BY NC SA license.