Monday, May 24, 2010

What Do Drivers Do In Computers?

I've been working on a project recently where I get to look at the kinds of questions people ask of Google, when phrased as a question.  For any given subject I get a glimpse of what people want to know, based on how they ask it.

Today's question is a very popular one on the subject of Windows device drivers :

What do drivers do in computers?
To really understand what drivers do in computers, you need to stop for a second and consider what's really going on under the covers.  You have "hardware".  That's the physical stuff - your memory, your CPU, your hard drive, your DVD player, your microphone.  Then you have "software".  That's the programs and files that live as bits (ones and zeroes) in the memory and on the hard drive.  The magic of how computers work could be summed up very briefly as, "The software tells the hardware what to do."

When you load a DVD, there are some physical steps.  You open the tray, usually by pushing a button.  You put the DVD in, and then close the tray either by pushing the button again or giving the tray a little shove.  And then your movie starts playing.  But that's where software comes in.  How exactly did the computer know that you put a new disk in the drive? The DVD player itself has just enough smarts to know when that tray door opens and closes. So when you close it, a signal is sent down the cable that connects player to motherboard. The CPU, which listens to everything going on around it, says "Ok, I just got an XYZ signal from the PDQ cable, what am I supposed to do with that?"

And that's where your "device driver" comes in.  The driver is a tiny little program whose sole purpose in life is to raise its little digital hand and say "Me!  Ooo!  I know how to handle XYZ signals from the PDQ cable!"  Ok, I'm being a little silly, but that's really what it comes down to.  Signals come through that cable, they go over to this little driver program, and the driver speaks a language that the rest of the software (Windows, typically) can understand.  So that XYZ signal is might translate into something like, "A new DVD has been inserted, pop up that dialog box that asks the user what she wants to do with it."

Before Windows, you actually would have had to do this all yourself.  When you bought a new printer or even a mouse, it would come with a disk that contained these drivers.  You'd copy them over to your hard drive, and then have to modify the startup programs to let the computer know these drivers were available.  One of the reasons that Windows took over the PC world was because, in the words of one famous industry analysis, it was just "a big bag of drivers."  That was the whole point -- now you don't have to deal with drivers anymore, they're just there.

Except when...they're not.  New devices come out all the time, it's impossible for Windows to have every driver for every device.  Worse than that, sometimes these driver programs have bugs in them and need to be updated.  So it's important to recognize the role these drivers play, and keep them updated.  Modern versions of Windows have an Update Service that will let you know when it's time to download new drivers.  Or you could look into a service like which will do all the work for you, scanning your computer to determine which drivers are out of date and then offering you an easy way to download them all in one visit.

Friday, May 21, 2010

Verizon DVR Hack : Deleting Multiple Episodes

If you've got Verizon DVR and you're like me, particularly if you've got the home media option for sharing between boxes, you'll often get 5 or so episodes of a tv show that you've watched but haven't deleted (because you watch them on other tv's where you can't delete).  And perhaps you've noticed a rather annoying issue with this where, once you're down to one episode, what happens is that it'll suddenly kick you back to the list of programs (because now there's no longer a folder for that show) and, because that episode is a few days/weeks old, it's buried at the bottom and you have to go find it.

Know what I'm talking about?  You go into the Grey's Anatomy folder, and delete one.  This is already complicated enough because for Verizon you need to hit like 4 buttons - Right arrow to get into folder, then Enter to select episode, then Down two times (or three, if you're half way through the program and have "Resume Play" option) to get to Delete, and then down for Yes, then enter.  That's alot of keystrokes.  So you have to do that for N shows.  And when there's that last show left, suddenly you have to do even more because now you have to scroll through your shows to go find where in time that last one is hiding.

Ok, enough of that, want to know the trick?  Skip the most recent episode.  Instead of deleting them top down, move to the second most recent one, and then delete them all.  What will happen when you only have the one most recent one left, it'll kick you back to the top level -- but look at that, your cursor is still sitting on the show you want.  Delete.

Monday, May 17, 2010

What We Need Is Psychic Search

Sunday afternoon, out having lunch at a noisy restaurant.  I hear part of a song I recognize, enough to say to my wife “Good song,” but then we can not hear the rest.  For the life of me I can not think of the song, and it drives me nuts all the way home.  Easy enough to Google lyrics, right?  Wrong, the sequence I heard was nothing but harmonizing, just background singers singing “Ahh ahh AHHHH….ahh ahh AHHHHH….ahh ahhh AHHHHHHH” where the third one has gotten higher, and then presumably it moves into the next verse which is what I can’t remember.

Try googling “song opening ahhh ahh ahhh” and see what happens. I got lots of hits for Led Zeppelin’s Immigrant Song, which I think is funny because it is both an accurate answer to the question, and also not even close to what I meant.  There were also lots of hits on modern dance club songs, but I’m old and don’t really know those enough to say one way or the other.

So I post on Facebook, and my friends begin guessing.  One says “At the Hop”, that old 50’s tune that starts “bahh bahhh bahhh bahhh, bahhh bahhh bahh bahhhh At the hop!  Well you can rock and you can roll and you can stomp and you can stroll at the hop….” (or whatever the words are).  Again, an accurate answer to the question pretty much, but not at all what I was looking for.

Undaunted I started googling for “Hum a few bars” hoping to turn up one of those search engines where you can sing into the microphone to guess your song.  I find nothing but hits from the 2006-08 era, none still active, so I’m wondering if I searched wrong.  I do try the iPhone app “Shazam”, but it can not recognize the tune so chances are I’m singing it too poorly.

I do strongly think that it is a 60’s tune, what I described to my friends as “hippie era” :).  Running out of ideas I google “summer of love music” and find a list of hits from that year (1967, I believe?)

And there it is.

Groovin, On A Sunday Afternoon by the Young Rascals.  My “ahh ahh AHHHH'” sequence is right there in the middle.

Now what I need is for somebody make the connection from my original query to the query that ultimately found me the answer.  We’ll make millions. :)

 UPDATE : Adding to my own post because the more I think about this, I think it’s not that far off.  I knew that it was a 60’s tune, right? Imagine if for my music search engine I was able to start there.  Just like the other folks I mentioned knew that there’s was a dance club song, and I knew mine was not.  From there maybe the engine could offer up some guiding questions like the tempo (this one is pretty slow), single or group (group), background harmony (definitely)…and so on.  I know that recommendation engines like Pandora break songs up into N different parameters and then match, saying “If you liked this song with these values then you’ll probably like this other song with similar values.”  But what if instead of doing that you offered up an interface where the user could just select those values on sliders?  “I want strong lead vocals, meaningful lyrics and a fast drum beat.” 

A slow 1960’s song with a harmonizing group certainly wouldn’t guarantee finding my song, but it would at least help narrow the search, and that’s huge. 

Wednesday, May 12, 2010

Variety in User Interfaces

Once upon a time Robert Heinlein wrote a book about a car with a voice synthesizer.  You talked to it, it talked back.  When it started speaking in phrases that the driver had never added, he chalked it up to one of his friends messing around with the database.  Eventually it got to the point where he wasn’t sure that the car wasn’t simply sentient.

I think that there’s lots of room for voice applications in today’s user interfaces.  Take the GPS in your car.  I expect that it talks.  You’re not supposed to read the screen while you’re trying to drive, right?  So let’s think about how it talks.

One way is to have a good solid database of what it can say, including names of all the streets.  That’s very handy, because instead of just “Turn left” it can say “Turn left on Quantum Street.”  But the downside is that you’ll be limited in other ways, and there’ll probably be very few ways that the computer phrases certain things.  It’s strength lies in being able to read you what is effectively a bunch of proper names.

The other approach to take is more generic, and not speak the street names.  “Turn left” and you rely on the layout of the road to know that you’re taking the left it wanted you to take.  Of course there is still an image on the screen, which can have the street name, so it takes half a second to look down and see whether you’re going where you are supposed to.

BUT! Once you substantially drop the size of the database by focusing only on a handful of “Turn left” / “Turn right” types of phrases, now you’ve opened the door to downloading your choice of voice.  You could have a celebrity read the 100 or so phrases, zip it up, and there you go.

Even better, you could be given instructions for how to do it yourself.  My TomTom has this, but I’ve never taken advantage of it.  I do have John Cleese from Monty Python doing my directions, but I’ve never sat down to record the kids and wife doing it (“Turn left here, turn left, here, turn left! You’ll kill us all!”).  The instructions were fairly complicated, explaining how you had to get each soundclip exactly the right length for it not to sound broken.  Forget that.

What I’d love to see, though, and this gets be back to my original story, is more variety.  The TomTom worked by saying “Ok, provide a folder consisting of 59 sound files labelled as follows.”  There’s a single sound clip for “end of trip”, so no matter how many times I use it, John Cleese is always there saying “You have reached your destination.  You may get out now, but I’m not going to carry your bags.”

What if instead of 59 sound files I was told to provide 59 directories, and in each directory there could be as many files as I wanted?  And whenever the device needed to play something it would instead just grab a random file from the appropriate directory?

That would be AWESOME.  For the most common ones you could sit down and record a good couple of dozen different ways to say it.  Maybe set something up that is user contributed so you just plain don’t know how many different sounds there are?  Or set a priority on them so that most of the time you get “Turn right” but once in a hundred you get “Turn right, moron.”

This approach, of course, would thus work for any device that talks back to you.  Your answering machine, maybe?

Type Type Revolution?

So, anybody know how to hack a “dance mat”?  I’ve  seen a bunch of games with this type of input device where you jump up and down on it – my kids even had a simple little “Strawberry Shortcake And Friends Dance Party” one if I recall.

Take one of those and build a game up around it where instead of dancing to music, you’re typing like you would on a cell phone. 

Improve your spelling, your accuracy…your appreciation of classic literature.  A game could be made out of it, where you have to answer questions but instead of jumping to A-D you have to spell out the answer.

I would totally buy that.  Somebody, if they haven’t already, needs to make a generic “dance mat” controller for Wii or something (balance board does not count!) that games can be written to.  Or maybe even a USB one so you can use it on the PC?

Tuesday, May 11, 2010

MongoDB Ruby Driver : nToReturn=100?

The other day I noticed in my logs that I kept getting my very large (6 million rows) result set back 100 rows at a time.  The log kept saying getmore, getmore, getmore… with nToReturn=100 set as a parameter.  So it was doing the right thing, but where was that 100 coming from?

A post on the user group gave me the answer – the default “batch_size” option for find is set at 100 in my version of the Ruby driver, when it should be set at o (which is the signal for the driver to just figure out the optimal size).  So you can override this by forcing batch_size:

@results = @objects.find(@conditions, {:batch_size=>0})

You should notice an improvement in performance.

Monday, May 03, 2010

Mongo Good.

Been working on an interesting project lately where I was handed some 6 million XML files with no meaningful schema, and told to make a warehouse out of them.  Each XML file also happens to have several hundred elements in it.  With no schema it would be a nightmare to try and make one, never fully sure whether you'd allocated enough space, whether just because your sample of this field yielded integers means that it always has to be an integer .. stuff like that.

I tried a very simple "Elements and Attributes" schema that is capable of loading any XML file.  Basically each Element and each Attribute get a row in the database.  This works fine for smallish data sets, but mine quickly blew up into billions of rows and was no longer manageable.

Then I stumbled over the "No SQL" movement, where we quite literally throw out decades of knowledge about how to set up a SQL schema of tables and rows, joins and where clauses .. and replace it with something more document-driven, no schema. This sounded perfect, since that's exactly what I had.

Enter Mongo DB.  With a little help from Ruby, my loader script for 6 million XML files now looked like this:

Foreach XML file:
  Load it via REST web service
  Parse it into a JSON object
  mongo.insert(object) Done!

Done!  Now if I'm hiding the name of a software vendor somewhere in the middle of one of those objects, like say {"machine"=>{"software=>[ "instance"=>{"name"=>"Microsoft something" ... }]}}  and I want to find them?  All I need to say is  find("" => /Microsoft/) and presto, I've got back my filtered list.  I can add a count() to get the total number of matches, or an each(...) or map(...) to process the list, or basically anything else I might want to do.  Of course I can also add an index on it to keep the performance high, too.

I'm finding this NoSQL stuff fascinating.  Takes a little while to mentally get around everything you've learned about normalizing a schema.  Instead of a dozen tables all crosslinked with each other, I've got one "collection" that is nested in a variety of ways.  But I quite literally spent 0 effort on schema design. 

I haven't even begun to touch on things like the built in sharding, which enables you to scale across multiple instances without having to worry about which server hold which portion of your data. Another player in this space, Cassandra (from the Facebook people), is most well known for being great at that. We looked at Cassandra but it had some schema setup required that I was not able to complete.  With Mongo I just inserted everything and now I'm free to query at will, no matter how complex the original key structure was.

I hope to write more here as I come up to speed on what is obviously the next big thing in the world of database design.