Monday, January 31, 2005

Automatic Meaning Discovery Using Google

Automatic Meaning Discovery Using Google

Abstract: We propose a new method to extract semantic knowledge from the world-wide-web for both supervised and unsupervised learning using the Google search engine in an unconventional manner. The approach is novel in its unrestricted problem domain, simplicity of implementation, and manifestly ontological underpinnings. We give evidence of elementary learning of the semantics of concepts, in contrast to most prior approaches. The method works as follows: The world-wide-web is the largest database on earth, and it induces a probability mass function, the Google distribution, via page counts for combinations of search queries. This distribution allows us to tap the latent semantic knowledge on the web.

and from the paper itself...
A comparison can be made with the Cyc project [14]. Cyc, a project of the commercial venture Cycorp, tries to create artificial common sense. Cyc’s knowledge base consists of hundreds of microtheories and hundreds of thousands of terms, as well as over a million hand-crafted assertions written in a formal language called CycL [20]. CycL is an enhanced variety of first-order predicate logic. This knowledge base was created over the course of decades by paid human experts. It is therefore of extremely high quality. Google, on the other hand, is almost completely unstructured, and offers only a primitive query capability that is not nearly flexible enough to represent formal deduction. But what it lacks in expressiveness Google makes up for in size; Google has already indexed more than eight billion pages and shows no signs of slowing down.

Sunday, January 30, 2005

Researchers Map The Sexual Network Of An Entire High School

Researchers Map The Sexual Network Of An Entire High School
COLUMBUS, Ohio – For the first time, sociologists have mapped the romantic and sexual relationships of an entire high school over 18 months, providing evidence that these adolescent networks may be structured differently than researchers previously thought.


Monday, January 24, 2005

testing ElementTree

Testing cElementTree...

from SimpleXMLTreeBuilder import *
import cElementTree as ElementTree

def xml2elem(xmlData):
p=TreeBuilder()
p.feed(xmlData)
return p.close()

if __name__ == "__main__":
text = """ root content
tag 1 text
content of tag2
deeper
tag2 tail

"""
elem=xml2elem(text)
print 'elem.Tag->', elem.tag # elem root
print 'findall->', elem.findall('tag1')[0].text # tag1 text
print 'interrogate second child...'
print ' ', elem.getchildren()[1].tag # tag2
print ' ', elem.getchildren()[1].items() # [('id', '2'), ('att2', 'val2')]
print ' ', elem.getchildren()[1].getchildren()[0].tag # next level gets... deeper
print 'for child -> elems children but not grandchildren'
for child in elem:
print ' ', child.tag #tag1 tag2 (but not deeper)

file=open('text.xml','w'); file.write('text'); file.close

print 'for elem in iterparse -> all levels...'
for event, elem in ElementTree.iterparse('test.xml'):
print ' ', elem.tag # #tag1 tag2 AND deeper


elem.Tag-> root
findall-> tag 1 text
interrogate second child...
tag2
[('id', '2'), ('att2', 'val2')]
deeper
for child -> elems children but not grandchildren
tag1
tag2
for elem in iterparse -> all levels...
tag
deeper
tag2
root


Wednesday, January 19, 2005

Mary Ellen Bates - Tip of the Month

Mary Ellen Bates - Tip of the Month:
Good tips...
"...try Google's synonym search. Add a tilde (~) at the beginning of the words child and obese (~child ~obese), and Google retrieves web sites that use any of those synonyms."
Google Personalized
Personalized Google is still in beta, but it's an interesting tool. Once you go to the Google Labs page and select Personalized, you will be sent to a new search page, that includes a link to [Create Profile]. You can specify the type of searching you typically do, ranging from biotech and pharmaceuticals to dentistry to classical music. Click [Save Preferences], and then type your search terms in the Google Personalized search box.


At the search results screen, you will now see something new -- a slider bar that lets you specify how much you want the search results sorted by those interests you specified. The default is minimal personalization; move the slider bar toward maximum, and you will see the search results change on the fly, as Google re-ranks the results based on your personal interests.


Specialized Searches
In addition to the well-known Google search tabs for searching the web, news and images, there are several specialized search tools for commonly-search subjects, including UncleSam for searching federal government information; University Search for searching within the sites of major colleges or universities;

Simplicity

dirtSimple.org:
"In my second professional programming job, I had a really interesting boss. When we had a design meeting, we would all sit around a whiteboard, and as Roger (my boss) threw out things we needed to accomplish, the other programmers and I would propose solutions, and Roger would say, 'Really? What if you just did X?', where X was some absurdly, ridiculously, jaw-droppingly simple thing.

Of course, X wouldn't always work; oftentimes one of us would find a hole in his idea. We'd all then try to fix the hole, but at some point the idea started to become too complicated for Roger's taste. 'How about Y?' Still ridiculously simple, and tantalizingly close to working.

Oftentimes, he 'cheated', by redefining the problem itself to make it a simpler problem to solve, or forcing the problem to fit some existing available solution. We would continue in this vein until the solution was so simple it hardly seemed like any work to actually implement, or it became absolutely clear that the problem would not yield to simplicity. In which case we simply packed it in for the day on trying to solve that problem, and we'd hit it again on another day."
The article continues, wonderfully, in this vein. I don't know that I'm as good as his boss, but I certainly recognize myself. Its reassuring to find there are others out there ...


Roger, by the way, was not a programmer by trade. He was a teacher, mostly of learning disabled children.

Saturday, January 15, 2005

infoSync World : Shake the Samsung SCH-310 to its senses

infoSync World : Shake the Samsung SCH-310 to its senses:

Airpen of the future?
"Samsung Electronics today unveiled the world's first mobile phone with built-in, 3-dimensional motion recognition technology. Dubbed the SCH-310, the handset calculates and ascertains movement in three dimensional space, and then carries out commands according to those calculations.

Samsung's new SCH-310
breaks new ground with
its 3D motion recognition
capabilities

Previously, several handset makers have shown prototypes drawing on motion recognition technology, however typically such concepts have relied on two-dimensional recognition rather than full spatial tracking. To enable the 3D motion recognition technology found in the SCH-310, Samsung devised a moving algorithm, resulting in applications for 22 domestic and foreign patents.

Predicting a future where 3D motion recognition technology will become an important user interface, Samsung envisions the technology revolutionizing mobile phone designs and doing away with the need for complex keypads on handsets, portable audio players, digital cameras and other handheld products. In particular, the company sees the technology changing mobile gaming.

The SCH-S310 relies entirely on 3D motion recognition technology for its user interface, enabling users to dial by 'writing' digits in the air. Shaking the phone twice will conclude a call or delete spam messages, whilst drawing the letters 'O' or 'X' will cause the handset to respond with 'yes' or 'no' voice messages. Also housing MP3 playback functionality, moving the phone sharply to the right will cause songs to skip forward; to the left, and it skips backward.
"

Schneier on Security: Physical Access Control

Very nice description of security logic applicable to software (not all that powerful for fences as commenters point out....

Schneier on Security: Physical Access Control: "In Los Angeles, the 'HOLLYWOOD' sign is protected by a fence and a locked gate. Because several different agencies need access to the sign for various purposes, the chain locking the gate is formed by several locks linked together. Each of the agencies has the key to its own lock, and not the key to any of the others. Of course, anyone who can open one of the locks can open the gate.

This is a nice example of a multiple-user access-control system. It's simple, and it works. You can also make it as complicated as you want, with different locks in parallel and in series."

Wednesday, January 05, 2005

ZDNet AnchorDesk: Why IM is so much better than e-mail

ZDNet AnchorDesk: Why IM is so much better than e-mail:

Curiously enough, multichat has none of these virtues.
"People I need to reach aren't responsive to e-mail anymore; they seem to check it every few hours or so, probably dreading the onslaught of spam and tedious threads that await them.

IM restores that rapid-fire pungency e-mail used to have, an electronic version of someone sticking their head in your office door.

I IM'd three of my IM peeps to find out why they like it.

Brian: Hey, I'm writing about IM in my next column. What's the essence of it?

Rafe Needleman (with his characteristically analytical take): presence is a key part of it. i know if you're there when i send a message to you. that's something you can't do either with e-mail or phone

Brian: OK, OK, I'll admit it, IM is cool But why?
Joni Blecher (with her usual larkish take): oh, easy--it's just like passing notes in class.

Brian: Brainstorming for my column: Why do you think IM is cool?
Stacy (my wife, as catty as a woman with five cats should be): you can be far more explicit than you can be on your work e-mail--i.e., bitching about your boss's awful new shoes w/o leaving a trace in the company's server logs. and one more a neat freak like you will appreciate: less crap to have to delete from your in- and out-box.
And this comment from a reader:
No latency
Its like a discussion, only you can't be interrupted.
And yet....

Monday, January 03, 2005

mod_pubsub

Why isn't this precisely what we need for Multichat?
mod_pubsub Project FAQ: "What is Repubsub?

Previous Mod-pubsub distributions (which are still available on Sourceforge) included a mod-perl server, a standalone Python server, client libraries in 10 programming languages, lots of sample code for the mod-perl server, random tools, badly-maintained docs, and the kitchen sink. However, we found that a large proportion of all the developers who wanted to use Mod-pubsub just wanted the Python server with the JavaScript client and were confused by the other stuff. So Repubsub is a minimal Mod-pubsub package comprised of the Python server and JavaScript client, rewritten for better performance and scalability.

Exactly how do Mod_pubsub's connections stay open?

Mod_pubsub's clients do not poll the server every so often. Instead, like Jabber, a Mod-pubsub server holds open the connection socket to its clients. The main difference between Mod_pubsub and Jabber is that we use standard HTTP as our wire format, whereas Jabber uses its own XML-based protocol. (Yes, we know that people think HTTP is a polling protocol... but it's not necessarily.)

Are there any other products like Mod_pubsub?

Similar projects include Pushlets, Jabber, and XMLHTTP. Unlike mod_pubsub, Pushlets can only maintain a few simultaneous connections, and require Java programming. With mod_pubsub, if you know how to use HTTP, you can do PubSub. Unlike Jabber, which uses its own XML-based protocol, we use standard HTTP. And Mod-pubsub is easier to work with, supports multiple users viewing the same data stream at the same moment, and is more flexible than XMLHTTP.
"

Form follows function. Architecture and technology follow form

Joel on Software - Don't Let Architecture Astronauts Scare You: "Your typical architecture astronaut will take a fact like 'Napster is a peer-to-peer service for downloading music' and ignore everything but the architecture, thinking it's interesting because it's peer to peer, completely missing the point that it's interesting because you can type the name of a song and listen to it right away.

All they'll talk about is peer-to-peer this, that, and the other thing. Suddenly you have peer-to-peer conferences, peer-to-peer venture capital funds, and even peer-to-peer backlash with the imbecile business journalists dripping with glee as they copy each other's stories: 'Peer To Peer: Dead!'

The Architecture Astronauts will say things like: 'Can you imagine a program like Napster where you can download anything, not just songs?'"

Sunday, January 02, 2005

Blogger Help : All about Blogger's post editor

Blogger Help : All about Blogger's post editor: "All about Blogger's post editor"

Shirky: Situated Software

Shirky: Situated Software: "We've been killing conversations about software with 'That won't scale' for so long we've forgotten that scaling problems aren't inherently fatal. The N-squared problem is only a problem if N is large, and in social situations, N is usually not large. A reading group works better with 5 members than 15; a seminar works better with 15 than 25, much less 50, and so on.

This in turn gives software form-fit to a particular group a number of desirable characteristics -- it's cheaper and faster to build, has fewer issues of scalability, and likelier uptake by its target users. It also has several obvious downsides, including less likelihood of use outside its original environment, greater brittleness if it is later called on to handle larger groups, and a potentially shorter lifespan.
"

Thursday, December 30, 2004

Oxfam America: Asian Earthquake & Tsunami

Oxfam America: Asian Earthquake & Tsunami

I just gave* to Oxfam, and put a link in Delicious. They seem to be in the forefront of aid to the region.

If you'd like to give as well, you can do so quickly and easily online. Just click on the link below:
http://www.oxfamamerica.org/whatwedo/emergencies/asian_floods_2004

* a pittance. 100,000 dead, 5 million in dire straits. A fraction of a penny per person.... But these things add up.

PubSub

PubSub: Testing PubSub

"Your subscription has been renamed.

It now appears in your Subscription Stack as Jon Schull's Weblog.

"

Thursday, December 23, 2004

Multichat

Big day for me. With a lot of help from my friends I finally got Multichat working on my laptop, put up a wiki page (static image below, for posterity) in the new Lab For Social Computing describing the system , and set up a project page for the team. I really think we're on to something.

Multichat

Multichat is a real-time variation on Tony Chang's Webnote http://www.aypwip.org/webnote/ (i.e., in our variant you see what people are typing as they type).

I'm running it on my personal computer, so if I'm online, please check it out. Chat me at jonschullaim, and I'll give you a URL and show you around. If I'm not online, you can get a partial feel for it from this:


The project was initially inspired by the need for a cross-platform, type-as-you-talk, multi-person chat program optimized to facilitate face to face conversations between deaf and hearing students in the classroom. (see First in Class Deafness Project at RIT).

But (thanks to the webnote foundation and to collaborator/coder Mike Axelrod) it turns out to be far more powerful than that. It will eventually work as a peer to peer system supporting conversations between any number of connected users (with or without access to the internet).

Moreover, we think we know how use it to add real time social "presence" to websites. (This variation is inspired by John Cappiello's wikalong http://wikalong.phunnel.org/wiki/ and the mind-blowing "Just Letters" http://web.okaygo.co.uk/apps/letters/flashcom/) So imagine a sidebar or companion browser window that can be yoked to an arbitrary URL in the main window. "Let's meet at the New York Times website," you might say and once there you will not only be able to see what people have written before, you'll be able to see what they are talking about in real time. ("Gee, I wonder what they're chatting about at slashdot..." you wonder), and whoever is loitering on that"street corner" will be visible to you.

Multichat is coded in python on the server side and javascript
on the client side. By the time we're done we'll be doing some
interesting things with java and streaming content as well. If you'd like to play along, let me know!!! It's open source, and the whole point is to create a widely-used social software application that is incidentally a wonderful tool for Deaf<->Hearing communication.

Collaborators, co-designers, co-developers, and co-conspirators include




Adding Optional Static Typing to Python

Adding Optional Static Typing to Python: Guido van Rossum writes, "Python's parser generator is so lame, but that in turn is intentional -- it is so lame to prevent me from inventing syntax that is either hard to write a parser for or hard to disambiguate by human readers, who always come first in Python's design.)"

Human's first. What a concept.

User-Driven Software Development as Science

User-drive software development as science

I found this paragraph quite interesting as a case study (in usability rhetoric) I haven't looked at the product yet. [paragraph breaks and bolding added]

Is EasyOffice a Microsoft Office (or WordPerfect Office or StarOffice) 'clone'?

No. Although EasyOffice contains most features common to Microsoft Office, as well WordPerfect Office and StarOffice, it was designed and functions as an original, innovative product.E-Press Corp. has in place a rigorous scientific research and engineering development program. This program is partially funded by the Canadian Federal Government, and undergoes a comprehensive scientific and financial governmental audit each year.

In this research program, the 'scientific method' is followed: hypotheses are generated, data is tested, results are analyzed, and new hypotheses are then generated, and so on. What this means in the case of EasyOffice, is that a particular version of EasyOffice is created. Then experimental subjects (some with no computer experience, and others with varying amount of expertise) are asked to create a particular document, a particular spreadsheet and a particular presentation using the experimental version of EasyOffice. The steps subjects take to do so, the difficulties they have, and of course, their successes are noted. The hypotheses are then considered in light of these results, and a new set of hypotheses, essentially a new version of EasyOffice, is then generated, and the procedure repeats.

The current version of EasyOffice essentially reflects this experimental origin, rather than a 'cloning' of any particular office suite. The bimodal success rates we obtained from inexperienced versus experienced users, is reflected in the 'Easy' and the 'Advanced' modes present in most EasyOffice programs. Many features which EasyOffice has (and indeed, Microsoft Office lacks) are there because of our observations of our experimental subjects. By the same token, many features that seem to reflect a Microsoft origin, are there also, but only indirectly so, because of demand of our experimental subjects. Experienced computer users are so entrenched with the Microsoft Windows and Office paradigm, that this group will independently cause a Microsoft-like version to emerge. This group vehemently rejects new ways of doing operations, new ways of organizing functions, and so on. On the other hand, inexperienced users are more open to new ideas, and actually do much better with a non-Microsoft-like paradigm. The effect of WordPerfect Office, StarOffice, Linux, or any other operating system or office suite, is found to be negligible in our studies in influencing subjects' outcomes.

Despite the skewing of our results due to the habituation (brainwashing?) of users with the Windows and other Microsoft paradigms, we have nonetheless maintained our vision of creating an innovative, intelligent, friendly office suite. Features that look similar in Microsoft Office, and WordPerfect Office, and StarOffice, may look similar in EasyOffice, but in many cases, our features are enhanced with the properties that subjects were found to need during our experiments. For example, although our 'File Menu' has become similar to Microsoft Office (and the other office suites), our 'File Menu' is bloated into 'File1' and 'File2' menus. Our subjects, were terrified of losing data, and would perform operations that reflected this fear. The many backup systems present in our 'File' menus, for example, reflect these observations, and allow our users an easier-use experience.

As well, while like other office suites, many of our features are built upon a dumb infrastructure, we are in the process of replacing many of the dumb routines with AI routines. As our mission statement says, "...rather than being a block of iron and silicon sitting on one's desk, it will become one's friend, in every sense of the word. You will talk to it, and it will talk back to you. It will empathize with you. It will do what you ask of it", and as the AI routines become incorporated into our software, this will happen.EasyOffice is not a clone of Microsoft Office, nor is it "one more office suite on the market". We believe EasyOffice is very special because it does what you, the user, want it to do.

Wednesday, December 22, 2004

Harvard Gazette: Academic turns city into a social experiment

Harvard Gazette: Academic turns city into a social experiment
Fascinating article

"One of former Bogota Mayor Antanas Mockus' many inspired strategies for changing the mindset - and, eventually, the behavior - of the city's unruly inhabitants was the installation of traffic mimes on street corners. (Photo courtesy of El Tiempo)
"

DrPython

DrPython
Dare I hope that this is a straightforward, nice looking open source python editor with code folding, and no show-stopper bugs? I do dare hope.

"DrPython is a highly customizable, extensible editor/environment for developing programs written in the Python programming Language. It is implemented in wxPython.
(wxPython is a set of python bindings (and extra widgets) for wxWidgets).
It uses Scintilla for the Text Controls.
"

Friday, December 17, 2004

This picture turns out to be a hoax...

Urban Legends Reference Pages: Inboxer Rebellion (Does Not Compute): "

Scientists from RAND Corporation have created this model to illustrate how a 'home computer' could look in the year 2004. However the needed technology will not be economically feasible for the average home. Also the scientists readily admit that the computer will require not yet invented technology to actually work, but 50 years from now scientific progress is expected to solve these problems. With teletype interface and the Fortran language, the computer will be easy to use and only
"