correct me if i’m wrong

In conjunction with utilizing Skribit, I am seeking ways in which I can devote more time to blogging with less of the overhead, such as reading articles, revising posts, linking to pages, and all the other timesinks of maintaining a blog. Since I will have a nice three-week winter vacation, I hope to blog daily and hence further incorporate it into a regular routine.

I sent an e-mail to a favorite blogger of mine, Daniel Tunkelang of The Noisy Channel, who was kind enough to post a detailed reply. Daniel is able to produce astonishing throughput without sacrificing quality and insight, especially for a non-professional blogger, who must answer to the demands of both a job and family. His main points of success are as follows:

• He blogs one hour each day of the workweek.
• His employer, Endeca, values personal blogging.
• Quick reactions to current events often produce great posts.
• Family and blogging are kept separate.

Perhaps the most important advice, which I am only coming to terms with now, intellectually, at 22-years old: 

Perhaps most importantly, cultivate a passion for unsolved problems, so that the world reminds you of them at every turn. It’s the same advice I’ve heard given to researchers, only that the threshold for publishing a blog post is a lot lower than that of submitting a publication for peer review.

There’s no doubt that Daniel, with a background at MIT and Carnegie-Mellon, two of the world’s leading educational institutions, has a lot to say. I hope to have such a vast amount of knowledge in the future, and I expect I can fast-track myself to such a position with access to the ever-more-ubiquitous information highway that has been inaccessible to previous generations. Additionally, blogging–a form of publishing that was also unknown before the Internet–helps put abstract thoughts into practice that simply aren’t thought-out and expressed in daily interaction with peers, and hence moves thoughts from the abstract to the concrete.

It’s unlikely that I have any regular readers of this blog, but for the one or two lost souls out there, you may have noticed a new feed on the right-hand panel of this blog: Skribit. This service “offers timely topic suggestions for bloggers to end writer’s block and assists bloggers in writing more relevant articles for their readers.”

I don’t have a shortage of blog topic ideas (rather the shortage is time, and how much I allocate toward blogging), but I thought it would be interesting if crowdsourcing the task of idea creation actually works. Further, since my goal is to make this blog discussion-based (read: “correct me if i’m wrong”), I welcome all user input. While I have a handful of blog ideas listed, they’re all posted by me, and I don’t expect to receive any without a large and dedicated reader base nor a well-defined niche.

Alas, if you’ve read this far, then I hope you do care about this blog to the extent that you’ll humor me with a suggestion. Tips: I enjoy writing about technology, software, education, and business.

I often find my fingers stumbling during long coding sessions as I Alt-Tab (or M-Tab in xmonad) to various windows I have open that reference documentation, Google searches, Stack Overflow, and the like. I may also have local documentation that I keep around in case I am coding off-line. Third, I may have a slew of buffers open in emacs in which I am referencing previously written code snippets that I need to incorporate into my current work. It’s overwhelming! I need information overload control.

I brainstormed what it would entail to consolidate the following items, and my current solutions for each:

• Access to updated documentation, blog postings, forum threads, and general help from others via the Web; Google.
• A way to easily organize and access code snippets; I started a blog for this, which probably wasn’t the best idea. I also use grep when applicable.
• Documentation for all your languages, frameworks, and tools; Google, Firefox bookmarks, and delicious.
• Unlimited access to standard algorithms and explanations for them; Java api, Python docs, and, most importantly, Wikipedia.
• A way to cross-reference language functionality (I know how to do this in language X, how is it done in language Y?); just an idea. This is always the most difficult when picking up a new language.
• A binary and scientific calculator; Google, and the built-in calculator for Windows or Ubuntu.

Ideally, this is accessible in on-line or off-line mode. The idea is similar to a full-featured IDE, but not. It’s not meant for coding, compiling, and sharing projects. It’s meant to be Alt+Tab-able to for speedy access. I envision some sort of functionality in which a user can cut-and-save snippets of information a la my friend Alex’s Snipd. This would most likely be done within a browser as a Gears website, caching any on-line content. Does any such application exist?

For the purposes of this post, I am going to call this idea “Project X.”

Right now, Google is basically serving as the gateway for most of the tasks Project X accomplishes. Obviously, Google is not intended for programming, but information retrieval. Project X is a vertical form of information retrieval for the specific task of programming. Google could certainly serve as a single component in the scope of X, especially if X is Web-based, allowing code snippets and other content found on Google to be grabbed, stored and organized in X.

Another example of a useful reference is Stack Overflow. I was recently converted to Stack Overflow when a question I had been seeking to answer for days was answered within five minutes, by two separate contributors. It is certainly filling a demand that was highly needed by the programming community–consolidation of Q&A for programming questions–but it is limited in it’s extent because of the service it’s intended to provide. It lacks the functional capabilities I cited in X (calculators, documentation, algorithm reference), but it can, like Google, be a single component in a larger toolset that X would provide.

Now, suppose we have consolidated a slew of diverse information in our person Project X database: code snippets in Java, Python and Haskell, documentation for Django, Python, the Java STL, MySQL, etc., along with perhaps some Quicksort optimizations and a graph of all the big O denominations. Bear with me here. What’s the best way of accessing this heterogeneous information?

Telepathy, of course. Pragmatically, I like the auto-suggest methodology that Google has made ubiquitous in Google Suggest via Search, Maps, Firefox, Chrome and others. Basically, type in what you want, and it will find it. Stack Overflow implemented this for when users ask a question with aplomb (in fact, Joel Spolsky has stated that this alone has eliminated many repeated questions from being asked in the first place. Source: unknown.). Suggest utilizes quite a lot of metadata for it’s search, not just restricting users to the main content of the search, but much more. I envision a similar utilization in Project X; add searchable tags to snipped content, search URLs for website-related content, search content categories, and more.

In fact, Project X could even go as far as having machines that crawl code blocks that users save and attempt to identify the language in which they were written. By extension, X could also extract useful tidbits by crawling all saved user data to find patterns related to key infromation and thus improve the retrieval process to make it “telepathic.” But this is probably overkill.

Any ideas? What would your ideal coding assistant entail?

EDIT: Ubiquity from Mozilla Labs demonstrates the exact paradigm I am describing. It utilizes general nformation sources, such as Yelp, Wikipedia, and Craigslist, while my idea is oriented towards programming support forums and documentation.

I spoke to Bill Hunt at the Silicon Valley Hackers and Founders meetup last night, CEO of StockMood. StockMood, a recent TechCrunch 50 finalist, is scouring the Web and determining whether the buzz around company X is hot or not. Based on this information, alongside trend data of company X’s stock, they hope to provide valuable insight to consumers and trading firms alike. Sign up for their beta–you’ll get invited promptly.

How are they gathering this information? Humans? That is so 20th century. Machines, of course! They are using sentiment analysis, an up-and-coming trend.* Sentiment analysis, according to Wikipedia, is (I have technical research papers, if you’re interested in more elaboration):

“…a broad (definitionally challenged) area of natural language processing, computational linguistics and text mining. Generally speaking, it aims to determine the attitude of a speaker or a writer with respect to some topic. The attitude may be their judgment or evaluation (see appraisal theory), their affectual state (that is to say, the emotional state of the author when writing) or the intended emotional communication (that is to say, the emotional effect the author wishes to have on the reader).”

This is very exciting news, indeed, as I expect this to be one of the many factors in an upcoming AI revolution. No revolution is without a complete list of players, of course, and StockMood isn’t the only one.** In fact, I recently blogged about a web application that psychoanalyzes blogs.

Collective Intellect, out of Boulder, CO, made StockMood’s exact value proposition approximately two years ago. However, they have moved toward becoming more of a marketing and analysis tool for companies based around that the buzz they get in social media applications.

Jodange (pronounced ye-dah-nj), is a NY-based spinoff of Cornell’s research in the area. They are a broad-based opinion collector that act as a search engine for sentiment. Check out their demo or even try their iGoogle gadget!

Happy Tweets analyzes the sentiment of a user’s recent tweets. It ranks each user from 100 to approximately 750 (the current score of the happiest tweep). Amusingly, I rank quite poorly with a score of 457:

The current Happyscore for abossy is:

457

Generally, people who have followed this person on Twitter lately will perceive this person as

Not Very Happy

While the author, Tim, claims to have built the website for his work in computational linguistics, this is a prime example of sentiment analysis. The two categories of research aren’t mutually exclusive, by any means, and have quite substantial overlap.

The beauty of such fields–as well as others that will partake in the AI revolution–is that they are largely invisible to the end user. This will result (as my Dad predicts) in a highly fragmented market with a huge multitude of niches, unlike the social networking and e-mail giants we have come to know (Facebook, Myspace, Gmail, Yahoo! Mail, etc.). This is due primarily to the network effect, where a service’s value is determined largely by the number of users, and each user being dedicated to only one or two services. The uses and value of sentiment analysis is still unclear, and Bill Hunt is certainly not trying to sell StockMood as a one-stop-shop, but as an additional tool for the arsenal of big traders.

I am eager to partake in the events as they unfold.

* Not enough of a trend for it to appear on Google trends, however. That doesn’t help my case at all.

** See a more exhaustive list than my few examples on the now-thrice-cited Wikipedia page on sentiment analysis.

EDIT: Added HappyTweet description. Added sentence about the network effect for clarity.

It’s coming.

Let’s see how long it takes for Apple to destroy it as they continue their reign of evil.

I have a number of blog posts that have been sitting in “draft” mode for several weeks. I can’t bring myself to “finish” them because the content seems old and the ideas stale. After all, news articles tend to have a 3-day lifespan (I wish I could find the Slashdot posting in which I read this!). Yet, I haven’t had any new thoughts, ideas or findings that have compelled me to blog.

Any tips?