I recently had an enlightening conversation with a brilliant friend that will remain nameless until he posts his own version of events. This friend had come across, somewhere on the internet, a page containing a list of “Twitter accounts that will follow you back, guaranteed”. As an experiment, he followed everyone on the list, and, as per the guarantee, they followed him back. Interestingly enough, however, is that he’s still racking up tons of followers, beyond those of the original list. Having more followers means you will get more followers.
Thinking about the ramifications of following what amount to dummy accounts in order to increase his follow rate, this friend pointed out that there’s no reason for him to unfollow the accounts he followed. Many of them don’t post, and those that do can be put into a list, and that list can be silenced. There’s no regulation in the Twitter TOS that says he can’t follow certain accounts (that would be counterproductive), and there’s no social stigma associated with it, as no one is likely to double-check his follow list for bots. To my friend, this seemed to be a fatal flaw in Twitter’s system—it can be easily gamed without consequence. Fortunately, Twitter is not a game. It’s a free-market economy.
In fact, Twitter is a very free economy. Because there are only the barest of regulations and developers can and do develop tools that automate and simplify actions in Twitter, users are left to rational self-interest as their only guide to navigating the economy, that is, their audience. Some, like my friend, do this, escaping from the trappings of social convention to further their clout by whatever means available. But because of this forced freedom, Twitter is a world of bubbles (as in “housing bubble”), where individual importance shifts from day to day, and where there seem to be more “experts” than regular Joes.
Compare this to Facebook—an economy with heavier restrictions, where businesses register differently than celebrities, who register differently than regular people. Where events aren’t just blipped out to the masses, they’re fully scheduled and organized. Where there’s such a thing as “like” and where it takes mutual consent to form a connection. Facebook, by its nature, abhors the expert, and embraces the commoner. Spamming is not only encouraged, but subsidized, and there’s no reason to aspire to any benchmark of social performance, unless you’re a company.
I count myself among what I imagine to be a majority of people, content with the shortcomings of both methods. We doubted the need for another major social media platform, especially after having been jilted by Diaspora. Yet there was the Goog, rolling out what at first blush was poised to be Wave 2.0. Thankfully, it wasn’t—it was an exceptionally simple design that approached the social media economy as a free market, much like Twitter. And, in fact, despite the brilliance that is the Circle system of separating one’s friends, there’s nothing yet baked into G+ that will keep it from becoming Twitter. There’s also nothing that can prevent it from becoming like Facebook. At the same time.
That’s the brilliance of Google+: giving the user the ability to change the rules. Because one can change one’s content stream like the channel on a television, users can make the service into anything they want: a Twitter clone, a Facebook clone (minus some largely unnecessary features), a mix of the two, or something else entirely. The killer app is being able to choose your audience, and thus, your economy.
A lot has been made of G+’s ability to succeed—whether it will be a functional social network on the order of Facebook and Twitter. The numerical evidence seems to indicate that that’s a big yes, but whether it will actually succeed—that is, yield an improvement for direct social media in general? That will depend on whether we, the users, learn to make it work to bridge the gaps in existing social media services. For that to happen, we need to experiment, and we need to share.
There’s a lot of upfront work that goes into building a social network, and in Facebook and Twitter, that setup seemed to be straightforward and repetitive. For Google+ users, the setup phase is ongoing, as we attempt to discover how best to format our circles and use them in a way that makes sense. Experimentation will yield functional formats on an individual level, but then each of us needs to continue to have a usability discussions with any friends willing to share, ensuring that disruptive ideas have a chance to spread through the network. I recommend a circle called “Meta” or “Google+” for that.
That’s really the unexpected joy of Google+: millions of people, working together to make something awesome.
Saturday, July 23, 2011
Monday, June 27, 2011
Writing About Technical Subjects (Pt. I)
So, you're a company intent on empowering people with hardware, software, and/or other new tech. Or you're a non-profit with Big Ideas about digital rights, tech literacy, or another 21st Century cause. Or you're a copywriter, learning to ply your craft in the content economy. Telling compelling stories about complex topics is exciting, but challenging. This fact is your greatest friend and most terrifying enemy:
Take Facebook. The second most-frequently visited place on the web is one of a handful of websites that has spawned its own verb. (When was the last time you said you were "facebooking"?) Behind the scenes, though, it's a complex messaging, image-sharing, networking beast of a site-slash-app, not to mention the developer interface for third-party apps.
The features aren't what Facebook splashes on the sign-up page, though. It says,
There's a simple principle in effect here: you don't sell features, you sell motivations. I've long appreciated market researcher and TED Talker Simon Sinek's verbiage:
Start With "Why"
That's hard and fast for me: successful pitchmen and copywriters have always done this, even if it wasn't worded exactly as above. That doesn't change in the world of software. In fact, in a world of rapid development and widespread innovation, there's even less place for people selling boxes of features.
Recently, Twitter posted a guide to help journalists and other newsy sorts use their service. They didn't have to—heaven knows people have already been using Twitter to do news since the dawn of tweets—but they did it as a help to their userbase. #TfN may not be copy, but it follows another guideline I love:
Solve a Problem
When possible, I present services as solutions to problems that people face. (As an aside, I think the use of "solution" as a buzzwordy substitute for "service", "application", or "product" dilutes the semantic power of that word.) Some companies don't use much text to show that they solve a problem—apps like Turntable.fm solve a problem (or perhaps "grants a wish") in an obvious way, and sell themselves on word of mouth alone. For everyone else, there's copy.
Trouble is, you have to explain a complicated problem like "T1 connections used to be state of the art, but nowadays you get more bandwidth if you sign up with a Wireless ISP" to someone who may not understand "T1", "bandwidth", or "Wireless ISP". If you're a Wireless ISP, that's going to cause problems.
The writer, then, has to make the problem and its solution simple enough to be clear to the reader. "Telling your story" has become a popular phrase in the marketing industry, and one of the tricks of the trade is just that: making copy into a story, with a conflict, a resolution, a hero, and—if necessary—a villain.
Tell a Story
Of course, you don't want to take storytelling to the extreme, either, or your reader will think you don't respect them. Leave some complex concepts in the narrative, and explain what must be explained.
Don't Condescend
Here's a bit of copy I wrote for a client that I think exemplifies these ideas:
For years, network service was fastest when it was delivered by physical cables and circuits, the “T1” being the most popular with businesses. The T1 was a workhorse, but, as content on the web has become more complex, it’s clear that T1 just can’t keep up. Fortunately, wireless technology has kept pace with increasing bandwidth needs—OneAxis.net wireless can offer speeds comparable to those of cable internet[...]
So, there you have it, an overview of copywriting on complex topics. Next up: News writing.
We now live in a world where the "simple facts" are not so simple.
Take Facebook. The second most-frequently visited place on the web is one of a handful of websites that has spawned its own verb. (When was the last time you said you were "facebooking"?) Behind the scenes, though, it's a complex messaging, image-sharing, networking beast of a site-slash-app, not to mention the developer interface for third-party apps.
The features aren't what Facebook splashes on the sign-up page, though. It says,
Facebook helps you connect and share with the people in your life.
There's a simple principle in effect here: you don't sell features, you sell motivations. I've long appreciated market researcher and TED Talker Simon Sinek's verbiage:
Start With "Why"
That's hard and fast for me: successful pitchmen and copywriters have always done this, even if it wasn't worded exactly as above. That doesn't change in the world of software. In fact, in a world of rapid development and widespread innovation, there's even less place for people selling boxes of features.
Recently, Twitter posted a guide to help journalists and other newsy sorts use their service. They didn't have to—heaven knows people have already been using Twitter to do news since the dawn of tweets—but they did it as a help to their userbase. #TfN may not be copy, but it follows another guideline I love:
Solve a Problem
When possible, I present services as solutions to problems that people face. (As an aside, I think the use of "solution" as a buzzwordy substitute for "service", "application", or "product" dilutes the semantic power of that word.) Some companies don't use much text to show that they solve a problem—apps like Turntable.fm solve a problem (or perhaps "grants a wish") in an obvious way, and sell themselves on word of mouth alone. For everyone else, there's copy.
Trouble is, you have to explain a complicated problem like "T1 connections used to be state of the art, but nowadays you get more bandwidth if you sign up with a Wireless ISP" to someone who may not understand "T1", "bandwidth", or "Wireless ISP". If you're a Wireless ISP, that's going to cause problems.
The writer, then, has to make the problem and its solution simple enough to be clear to the reader. "Telling your story" has become a popular phrase in the marketing industry, and one of the tricks of the trade is just that: making copy into a story, with a conflict, a resolution, a hero, and—if necessary—a villain.
Tell a Story
Of course, you don't want to take storytelling to the extreme, either, or your reader will think you don't respect them. Leave some complex concepts in the narrative, and explain what must be explained.
Don't Condescend
Here's a bit of copy I wrote for a client that I think exemplifies these ideas:
For years, network service was fastest when it was delivered by physical cables and circuits, the “T1” being the most popular with businesses. The T1 was a workhorse, but, as content on the web has become more complex, it’s clear that T1 just can’t keep up. Fortunately, wireless technology has kept pace with increasing bandwidth needs—OneAxis.net wireless can offer speeds comparable to those of cable internet[...]
So, there you have it, an overview of copywriting on complex topics. Next up: News writing.
Friday, June 10, 2011
Intersection: Trumor + The Filter Bubble
First in a series of pieces that examine the hidden links between apparently unrelated ideas.
Scientists at the Laboratory for Information and Decision Systems at MIT have been working on a way to determine how far an idea will propagate via Twitter. The system, named Trumor, measures the reach and influence of individual users on Twitter, much like the commercially-available service, Klout. Trumor, however, creates a list of "superstars" in each topic (e.g. soccer, automobiles, copyright law). These users are very likely to have their comments propagated through the network.
The Filter Bubble is the title of a book and the name of an idea developed by online activist Eli Pariser. The basic premise is this: online information is being increasingly organized automatically by individual preference, and this algorithmic curation may not be to our benefit. As an example, Pariser notes the results pages for Google searches two of his friends did on "Egypt" during the revolution earlier this year. One friend received mostly news, the other received travel agencies and basic encyclopedia information, but nothing on the protests. As human beings rarely like their ideas to be challenged, the filter bubble tends toward increasing divisions between groups of people, based on ideology.
The Intersection: As people self-organize by ideology, they will likely either choose to restrict the number of information sources they rely on, or the filter bubble will do that for them. As the bubble closes in, information from outside will leak in less and less frequently, making the power of online thought leaders very strong as concerns influencing those that follow them. (Incidentally, the implications of the word "follow" in the Twitter context will become creepier.)
As this occurs, the knowledge of who influences whom (will MIT retain this information? will it be made public?) will likely fall into the hands of corporations, political parties, hackers, and activist groups. The game will become "Who influences the influencers?" as groups attempt to convince members of the superstar list they're interested in to carry their content. This structure of top-down siloed syndication would be very weak to subtle subversion and account hacking.
In sum: As the world outside our senses becomes more important, our reliance on others' perspectives increases. As our filter bubbles close in, our worldviews become less robust and more vulnerable to manipulation via the social media superstars we rely on. Reality-hacking and reality-selling are imminent, but not in a cool cyberpunk way.
Scientists at the Laboratory for Information and Decision Systems at MIT have been working on a way to determine how far an idea will propagate via Twitter. The system, named Trumor, measures the reach and influence of individual users on Twitter, much like the commercially-available service, Klout. Trumor, however, creates a list of "superstars" in each topic (e.g. soccer, automobiles, copyright law). These users are very likely to have their comments propagated through the network.
The Filter Bubble is the title of a book and the name of an idea developed by online activist Eli Pariser. The basic premise is this: online information is being increasingly organized automatically by individual preference, and this algorithmic curation may not be to our benefit. As an example, Pariser notes the results pages for Google searches two of his friends did on "Egypt" during the revolution earlier this year. One friend received mostly news, the other received travel agencies and basic encyclopedia information, but nothing on the protests. As human beings rarely like their ideas to be challenged, the filter bubble tends toward increasing divisions between groups of people, based on ideology.
The Intersection: As people self-organize by ideology, they will likely either choose to restrict the number of information sources they rely on, or the filter bubble will do that for them. As the bubble closes in, information from outside will leak in less and less frequently, making the power of online thought leaders very strong as concerns influencing those that follow them. (Incidentally, the implications of the word "follow" in the Twitter context will become creepier.)
As this occurs, the knowledge of who influences whom (will MIT retain this information? will it be made public?) will likely fall into the hands of corporations, political parties, hackers, and activist groups. The game will become "Who influences the influencers?" as groups attempt to convince members of the superstar list they're interested in to carry their content. This structure of top-down siloed syndication would be very weak to subtle subversion and account hacking.
In sum: As the world outside our senses becomes more important, our reliance on others' perspectives increases. As our filter bubbles close in, our worldviews become less robust and more vulnerable to manipulation via the social media superstars we rely on. Reality-hacking and reality-selling are imminent, but not in a cool cyberpunk way.
Monday, May 23, 2011
Reddit, the Abortionplex, and the Infinite Rabbit Hole
Fake news, people that think it's real news, and people that want people to know that there are people who thought the fake news was real news.
On the 18th, The Onion ran an article detailing a new $8 billion Planned Parenthood facility called "The Abortionplex". In typical Onion fashion, the article was gleefully tongue-in-cheek, timely, and completely, utterly false. The Onion is, of course, a satire news site.
That doesn't necessarily stop people from believing it. This wouldn't be the first time that the masses have bought an Onion line. Nor would it be the most ridiculous thing the general populace has believed. That being said, this graphic (click through) shouldn't come as a surprise to anyone—and yet it's one of the most interesting internet artifacts I've seen, as there is no attempt at all to prove that it is authentic.
I want to be sure that I go one record saying that I don't have any doubts as to the legitimacy of the graphic. However, if someone wanted to create a false graphic on this or any other theme, the ubiquity of Facebook thumbnails and Photoshop makes it possible for them to get involved in made-up guerrilla journalism. The Culture Wars no longer require facts (but they do need the lulz).
Eli Pariser notes the advent of the "filter bubble": the way that Google, Facebook, et al. filter content that you receive based on previously expressed preferences. His TED Talk approaches the most obvious problem with the filter bubble, which is that as websites tailor content automatically, people aren't exposed to opposing viewpoints. There's another angle here, which is that these filter bubbles protect false ideas from being debunked except by untrusted "outsiders". It's clear that the Obama Kenyan birth certificate is a fake, that vaccines do not cause autism, and that the Bush administration did not conspire to cause 9/11, but these ideas continue to spread.
Consider this: we are creating a world in which large social groups not only have different views as to how the world works, but can actively produce proof that their worldview is correct. Worse than that, the low barrier-to-forgery makes it easier and easier for us to disregard facts we find inconvenient to our preconceptions.
On the 18th, The Onion ran an article detailing a new $8 billion Planned Parenthood facility called "The Abortionplex". In typical Onion fashion, the article was gleefully tongue-in-cheek, timely, and completely, utterly false. The Onion is, of course, a satire news site.
That doesn't necessarily stop people from believing it. This wouldn't be the first time that the masses have bought an Onion line. Nor would it be the most ridiculous thing the general populace has believed. That being said, this graphic (click through) shouldn't come as a surprise to anyone—and yet it's one of the most interesting internet artifacts I've seen, as there is no attempt at all to prove that it is authentic.
I want to be sure that I go one record saying that I don't have any doubts as to the legitimacy of the graphic. However, if someone wanted to create a false graphic on this or any other theme, the ubiquity of Facebook thumbnails and Photoshop makes it possible for them to get involved in made-up guerrilla journalism. The Culture Wars no longer require facts (but they do need the lulz).
Eli Pariser notes the advent of the "filter bubble": the way that Google, Facebook, et al. filter content that you receive based on previously expressed preferences. His TED Talk approaches the most obvious problem with the filter bubble, which is that as websites tailor content automatically, people aren't exposed to opposing viewpoints. There's another angle here, which is that these filter bubbles protect false ideas from being debunked except by untrusted "outsiders". It's clear that the Obama Kenyan birth certificate is a fake, that vaccines do not cause autism, and that the Bush administration did not conspire to cause 9/11, but these ideas continue to spread.
Consider this: we are creating a world in which large social groups not only have different views as to how the world works, but can actively produce proof that their worldview is correct. Worse than that, the low barrier-to-forgery makes it easier and easier for us to disregard facts we find inconvenient to our preconceptions.
Friday, May 6, 2011
The Tolle Paradox: How Buzzwords Kill Big Ideas
Whether you love, hate, or don't know Eckhart Tolle, he's an important guy in modern thought. He's been on Oprah's show multiple times (which is really the modern touchstone of influence), his books are bestsellers, and the London-based spirituality group Watkins Review rates him the most spiritually influential person in the world. (Note: The author of this article is largely not convinced of the value of Tolleanism in general, but will not deny his importance.)
You may have heard some of his principles. The two pillars of his work are 1) the rejection of humanity's tendency to live in the past or the present, and 2) the idea of experiencing ideas and objects without categorizing them. That is, experiencing a flower or a day at the beach as an object and a time period, respectively, without assigning them "labels".
You may have even heard "labels" as a negative (which predates Tolle, but has a new meaning in his context)—"don't 'label' this moment," and the like. You probably have heard people ask you to be "present in the moment". Unfortunately, these shortcuts deconstruct the entirety of Tolle's thought. By using labels to describe the act of not using labels, the entire purpose of separating the mind from the self is defeated. The buzzwords kill the idea. The buzzwords become a cudgel by which any number of ideologies are forced upon an audience.
Why is this? I don't have any studies, but let me posit a theory: Buzzwords deconstruct by their nature. Thinking sorts spend months and years on their frameworks, working out exactly what they mean, and running their words through editors and publishing houses and colleagues before releasing them to the public. Their disciples struggle, but eventually work out a practical way to implement their words. Then the work becomes famous, and there arrive on the scene people who want the power of the movement without its substance.
It's easy to learn the jargon of a philosophical movement, then use this new language to identify yourself as sympathetic to the movement itself. One such an intruder overlays his or her own personal philosophy over the jargon of the movement, a weapon of ideology, the buzzword, has been fashioned—it's a funnel by which people can be moved from one school of thought to another without recognizing the shift.
Unfortunately, it's not just something that happens in the ethereal world of new age spirituality. Agile software development is a method for developing applications that has changed the face of coding. The basic principle is this: One builds out fully-formed features of software in small time increments. Every iteration of the process yields a feature for the application that could be brought to market by itself.
Agile, like Tolleanism, is widely accepted and preached, and is mostly harmless. No one would really take issue to focusing on present experience, nor on building software on a marketable piece by marketable piece basis. However, in both cases, powerful rhetoric gets changed into a tool that serves its wielder's interest only. Being asked not to "label" things becomes learning not to talk back to your boss; a "scrum" becomes a sweatshop.
Those burned by the tool-wielders then tend to backlash against Tolle and against Agile, without realizing they've been had by masters of narrative engineering.
You may have heard some of his principles. The two pillars of his work are 1) the rejection of humanity's tendency to live in the past or the present, and 2) the idea of experiencing ideas and objects without categorizing them. That is, experiencing a flower or a day at the beach as an object and a time period, respectively, without assigning them "labels".
You may have even heard "labels" as a negative (which predates Tolle, but has a new meaning in his context)—"don't 'label' this moment," and the like. You probably have heard people ask you to be "present in the moment". Unfortunately, these shortcuts deconstruct the entirety of Tolle's thought. By using labels to describe the act of not using labels, the entire purpose of separating the mind from the self is defeated. The buzzwords kill the idea. The buzzwords become a cudgel by which any number of ideologies are forced upon an audience.
Why is this? I don't have any studies, but let me posit a theory: Buzzwords deconstruct by their nature. Thinking sorts spend months and years on their frameworks, working out exactly what they mean, and running their words through editors and publishing houses and colleagues before releasing them to the public. Their disciples struggle, but eventually work out a practical way to implement their words. Then the work becomes famous, and there arrive on the scene people who want the power of the movement without its substance.
It's easy to learn the jargon of a philosophical movement, then use this new language to identify yourself as sympathetic to the movement itself. One such an intruder overlays his or her own personal philosophy over the jargon of the movement, a weapon of ideology, the buzzword, has been fashioned—it's a funnel by which people can be moved from one school of thought to another without recognizing the shift.
Unfortunately, it's not just something that happens in the ethereal world of new age spirituality. Agile software development is a method for developing applications that has changed the face of coding. The basic principle is this: One builds out fully-formed features of software in small time increments. Every iteration of the process yields a feature for the application that could be brought to market by itself.
Agile, like Tolleanism, is widely accepted and preached, and is mostly harmless. No one would really take issue to focusing on present experience, nor on building software on a marketable piece by marketable piece basis. However, in both cases, powerful rhetoric gets changed into a tool that serves its wielder's interest only. Being asked not to "label" things becomes learning not to talk back to your boss; a "scrum" becomes a sweatshop.
Those burned by the tool-wielders then tend to backlash against Tolle and against Agile, without realizing they've been had by masters of narrative engineering.
Friday, April 29, 2011
Getting a Celebrity to Tweet to You
A well-timed Twitter post got Dulé Hill to send me one in return. You know that's right.
The other day I had an idea. A goal, really, albeit a small one. I had found that there were a couple of celebrities on my Twitter feed that seemed to be very interactive with their followers, and I thought, "I should get one of them to interact with me!" Now, I don't want you to take me for one of those celeb-stalker sorts...I follow one actor, one athlete, one novelist, and a handful of journalists.
The two celebs I follow that I deemed "interactive" couldn't be more dissimilar, but, strangely, both are in Vancouver. William Gibson, father of cyberpunk and author of some truly awesome books; and Dulé Hill, best known for his role as Charlie Young on The West Wing and Burton "Gus" Guster on USA's Psych, both of which are excellent television series.
Gibson, for his part, seems to like to tweet about current events, largely retweeting funny and interesting things he sees, but he responds when people ask him questions. Hill tends to converse with the crowd constantly, and posts about friends, especially West Wing and Psych castmates.
After a failed attempt to get Gibson to respond, I decided to focus on Hill. Now, when I say "focus", I mean, in the immortal words of Mr. Hill's castmate James Roday, "Wait for iiiiiiit!" And the moment arrived.
I saw that Rob Lowe, formerly of The West Wing was trending at the time, due to his appearance on Oprah. Hill had tweeted a number of times regarding his admiration for Lowe, so my tweet was simple, "@dulehill Rob Lowe is trending right now. Just thought you should know." Hill's response? "@benchatt Nice!" That was it. Mission accomplished.
Of course, I told everyone on Facebook what just happened on Twitter. Yes, that sentence is as strange to me as it is to you. The funny thing is that my one-word brush with greatness has stayed on the web—I think I've mentioned it once in real life, to my wife, who then probably saw it on Facebook anyway.
Social media may bring us into closer contact with celebrities these days, but those brushes with fame aren't nearly as cool as running into Ralph Macchio at Taco Bell.
The other day I had an idea. A goal, really, albeit a small one. I had found that there were a couple of celebrities on my Twitter feed that seemed to be very interactive with their followers, and I thought, "I should get one of them to interact with me!" Now, I don't want you to take me for one of those celeb-stalker sorts...I follow one actor, one athlete, one novelist, and a handful of journalists.
The two celebs I follow that I deemed "interactive" couldn't be more dissimilar, but, strangely, both are in Vancouver. William Gibson, father of cyberpunk and author of some truly awesome books; and Dulé Hill, best known for his role as Charlie Young on The West Wing and Burton "Gus" Guster on USA's Psych, both of which are excellent television series.
Gibson, for his part, seems to like to tweet about current events, largely retweeting funny and interesting things he sees, but he responds when people ask him questions. Hill tends to converse with the crowd constantly, and posts about friends, especially West Wing and Psych castmates.
After a failed attempt to get Gibson to respond, I decided to focus on Hill. Now, when I say "focus", I mean, in the immortal words of Mr. Hill's castmate James Roday, "Wait for iiiiiiit!" And the moment arrived.
I saw that Rob Lowe, formerly of The West Wing was trending at the time, due to his appearance on Oprah. Hill had tweeted a number of times regarding his admiration for Lowe, so my tweet was simple, "@dulehill Rob Lowe is trending right now. Just thought you should know." Hill's response? "@benchatt Nice!" That was it. Mission accomplished.
Of course, I told everyone on Facebook what just happened on Twitter. Yes, that sentence is as strange to me as it is to you. The funny thing is that my one-word brush with greatness has stayed on the web—I think I've mentioned it once in real life, to my wife, who then probably saw it on Facebook anyway.
Social media may bring us into closer contact with celebrities these days, but those brushes with fame aren't nearly as cool as running into Ralph Macchio at Taco Bell.
Monday, April 25, 2011
Finding the Weird Words (Vocab and Gmail Corpus Part I)
Today, I can finally begin to analyze the text of my Gmail Chat corpus.
My first thought is that IM text is probably somehow different from speech and pre-planned text: books, articles, letters, etc. After the rather painful process of piecing together Perl scripts to divide my corpus between the text I wrote and the text other people wrote, I ran those two files through Laurence Anthony's great AntWordProfiler, which compares a text against the General Service List and Academic Word List.
The GSL and the AWL are general vocabulary lists designed to help ESL teachers give learners of English the most useful vocabulary first. While English has one of the largest lexicons (if not the largest) in the history of language, about 80% of most texts are comprised of a few thousand very common words. The 2000 "most useful" of these words are the GSL. The AWL adds words that appear in basic academic texts and newspapers.
AntWordProfiler divided the words from my texts into four categories: those that were found in the first thousand words of the GSL, those that were found in the second thousand, those that were found in the AWL, and those that were not found in either the GSL or the AWL. While not completely accurate, one might consider the results as being divided between "very frequent", "frequent", "somewhat frequent", and "not found".
This is the distribution of the four groups, divided in two between words typed by others and words typed by myself. I ended up using a lower percentage of K1 (first thousand) GSL words, and a higher percentage of off-list words. This is probably because my corpus of interlocutors' words includes more than 80 different people, of varying word-choice preference. As the average should be 80% K1+ K2 words, neither I nor my associates are more loquacious than the average bear. (Not true: Average bear K1 is probably 0; most bear speech is "Not in List".)
What I find most interesting, really, is how similar the two breakdowns are. The difference between the percentages of GSL K2 and AWL were a matter of fractions of a percent each, and both corpora show distributions predicted for normal English text, which indicates that instant messaging text is no different from speech or pre-planned text in its word choice (as opposed to, say SMS text, which almost certainly has a different distribution).
Next up: What was in that "Not in List" group? A look at my weird words.
My first thought is that IM text is probably somehow different from speech and pre-planned text: books, articles, letters, etc. After the rather painful process of piecing together Perl scripts to divide my corpus between the text I wrote and the text other people wrote, I ran those two files through Laurence Anthony's great AntWordProfiler, which compares a text against the General Service List and Academic Word List.
The GSL and the AWL are general vocabulary lists designed to help ESL teachers give learners of English the most useful vocabulary first. While English has one of the largest lexicons (if not the largest) in the history of language, about 80% of most texts are comprised of a few thousand very common words. The 2000 "most useful" of these words are the GSL. The AWL adds words that appear in basic academic texts and newspapers.
AntWordProfiler divided the words from my texts into four categories: those that were found in the first thousand words of the GSL, those that were found in the second thousand, those that were found in the AWL, and those that were not found in either the GSL or the AWL. While not completely accurate, one might consider the results as being divided between "very frequent", "frequent", "somewhat frequent", and "not found".
This is the distribution of the four groups, divided in two between words typed by others and words typed by myself. I ended up using a lower percentage of K1 (first thousand) GSL words, and a higher percentage of off-list words. This is probably because my corpus of interlocutors' words includes more than 80 different people, of varying word-choice preference. As the average should be 80% K1+ K2 words, neither I nor my associates are more loquacious than the average bear. (Not true: Average bear K1 is probably 0; most bear speech is "Not in List".)
What I find most interesting, really, is how similar the two breakdowns are. The difference between the percentages of GSL K2 and AWL were a matter of fractions of a percent each, and both corpora show distributions predicted for normal English text, which indicates that instant messaging text is no different from speech or pre-planned text in its word choice (as opposed to, say SMS text, which almost certainly has a different distribution).
Next up: What was in that "Not in List" group? A look at my weird words.
Subscribe to:
Posts (Atom)