| Home > Technology > Why Apple's Spotlight technology is more important than you might think |
| Why Apple's Spotlight technology is more important than you might think | | Date Created: 29 Nov, 2004, 03:41 PM |
 The end of the calendar year is usually the time the technology pundits emerge to predict what the next year has in store for us.
Last year, I had a go and predicted for my luddite psychology colleagues that Voice over Internet Protocols (VoIP) would be a technology to watch as well as video-conferencing. No elephant stamps for that bit of crystal ball gazing.
I want to suggest that for 2005, two domains will "emerge and merge" and make huge changes for how we use information technologies. There have already been large changes occurring as a result of their emergence in the last couple of years, but next year - with the introduction of Apple's Spotlight in 10.4 "Tiger" - I expect these changes will drift down from the early adopter set into mainstream use.
The changes that will merge are the rapid increase in storage - both absolute size and unit price per MB - and the new technologies underlying desktop (and laptop) search.
Each is being driven from independent domains, but there is a clear convergence happening. Just like the development of the 1" hard drive enabled the iPod to emerge as the iconic mobile music storage system, allowing users the hitherto unthought-of ability to carry with them always their entire music collection, the extreme rapidity with which storage capacity has grown will spawn new developments, especially in search technologies.
The iPod of course has led a double life, initially as a music player, and as a storage device. In the Mac version the ability to act as an emergency tool allows users to start up from the iPod to do diagnostics and repairs on their Powerbooks when (rarely) needed. (Ref: Go here to see how prophetic yet also wrong Robert Accetura at MacVillage.net was in a post just two days after the iPod was first announced in 2001).
The iPod's hardware allowed for very large storage capacity, while its software permitted the rapid sorting and retrieval of desired information: locating and playing sound tracks. Note that I used sound, since the iPod also spawned unpredicted uses such as podcasts and audioblogs.
Yes, yes - any mp3 player could have done the same, but they didn't. The combo. of iTunes/iPod/RSS feeds and motivated programmers on the Mac platform intially got that ball rolling. Which is why it now has the generic "podcast" label, not WiMPcast.
The iPod represents one of the significant changes in data management whereby developments in hardware and software have invoked shifts in how we think about music and our daily behaviours. Many readers probably now daily listen to their iPods or mp3 players in their cars, planes, in queues, on buses and trains, and even at work, especially if they are listening to informative podcasts, such as Doug Kaye's ITConversations.
Storage capacity obeys its own version of Moore's law. Look at how the price of hard drives and USB thumb drives has lowered in the last 12 months for a given size. Can you easily locate a 16MB thumbdrive? Sooner or later, they'll appear as give-aways in cereal boxes, and most trade shows also give them away in exchange for a little market research from you. 256MB and 512MB are the new 16 and 32MB sizes, and now the same price within 12-18 months after we first saw thumbdrives appear. Underestimate storage needs of your users and it will come back and bite you as PalmOne has discovered with its Treo 650 memory upgrade issues.
Similarly, hard drive capacity means premium laptops will emerge in 2005 with 100GB drives, and 160GB drives are now being released in Windows Media Centre laptops.
The days of "dumping" files to back-up media for the sake of making room for new files will soon pass, and hoarding files - pdfs, docs, jpgs, websites, trialware, IM conversation histories, bit torrented movies, etc. - will becomes the norm. The number of potentially redundant files will be in direct proportion to a hard drive's free space.
Those of compulsive personality type will be in heaven, allowing their hoarding habits to reach new heights. The rest of us, perhaps too lazy to do a regular spring clean until our operating system begin to slow down with the detritis of thousands of files, will confront a new dilemma: maintaining a proper filing system so we know where everything is, allowing us to speedily access a file we "know" is on the computer.
Both Windows and Mac OS have reasonable global search facilities. Like Google, you can be a power searcher, or you can take the simple route and expect an enormous number of irrelevant "returns" to your search. There are a variety of indexing applications to augment your search, and such technology was available to me on the Mac a decade ago with the likes of ON Location, and UltraMind, now gone to software heaven.
New search technologies have already emerged to seek specialised blogs, podcasts, RSS and Atom feeds, news headlines, email, etc. Take a look at MSN's Sandbox to see what Microsoft is up to.
Google first showed just how important and valuable seach technology can be. Offer an easy no-brainer interface, rapid return of results, a growing assortment of search specialities, such as images, language variations, and scholarly articles, and you can show the average user you don't need to be a Library Science major to do the sort of search even they would have considered impressive just a few years ago.
Delivering professional quality performance and features into the hands of rank amateurs is surely a technological Holy Grail, and Google, Apple, and Microsoft have been key drivers in this area.
In order to get its next OS, Longhorn, to market more quickly, Microsoft appears to have put to one side, albeit temporarily, its search engine, known as WinFS. This surprises some of its independent commentators, such as Paul Thurrot who blogged:
"Since last fall, Microsoft has been preaching the three core pillars of Longhorn as being the WinFS storage engine, the Avalon presentation layer, and the Indigo Web services platform."
MS of course is vitally interested in Search and Storage technologies as its Sandbox page suggests. Also visit Jim Gray's MS website to see what he has placed in the public domain as fruits of his department's labours, and where MS is headed especially in turning the Internet into the leading Astronomy application.
Another important IT contributor, MS research Fellow Gordon Bell, is the guinea pig for the MyLifeBits project, aiming to record everything he does, sees and feels for the remainder of his life.
Here's what Wired magazine said:
"Inspired by Vannevar Bush, an engineer who in 1945 posited the creation of a "configurable storehouse of knowledge," Bell started scanning material in 1999. He has amassed more than 10 gigabits -- or 7,000 floppies -- of information. Some of it is available on his website.
"My massive regret is that I don't have everything," said Bell, who has tossed the hard copies of almost all of his paper documents.
Researchers predict that in five years, the price of terabyte hard drives will dip below $300, allowing for nearly endless recording. With a terabyte of storage, users could upload over 3.6 million 300-kilobyte images, for example, or 290 hours of video.
...One of Bell's greatest challenges is how to search through such a massive database. Even as our lives become more digitized, there still is no easy way to coherently organize the pictures, e-mails and documents that form the stuff of our existence. Researchers still must know the file name or the date of creation for image and audio files, for example.
Someday, MyLifeBits will allow people to Google their own lives, Bell says."
Over at ITConversations, one of Dave Winer's Bloggercon III discussion mp3s for Newbies had people discussing why they blog. One woman said she was amazed that her blog diary and her Flikr photo album allowed her to recall what she had done on certain days, and indeed how she looked: her hairstyle, clothes, etc.
Now that might sound uninteresting to some, but who knows when that sort of information could be vitally important? And that's the point - we don't know when data will become important to us.
Those who become expert knowledge managers (that is, they bring information into action and cause changes to occur) are - my guess - those who have intuitively made records of data they "know" they need to record, even if they have no specific need for it at the time. Go read Richard Branson's autobiography (Losing my Virginity) for his idea-notation techniques to see one variation on the theme.
I just emailed to my friend, writer and journalist Charles Wright, that on a visit to his Bleeding Edge Skunkworks, I can recall him saying that what kept him chained to the Windows platform as his professional installation (ie, from which he earns his income) was a very simple text retrieval application (InfoSelect). It let him rapidly search his hard drive for articles he had written or those of others he had stored as text. It was the deal breaker for him at a time when OS X was not really useful as a desktop OS. InfoSelect is now in v.7 and not quite the same simple app. it was when I first saw Charles use it.
I wrote to him asking if he, like me, ever purged his email inbox, or download folder, only to locate an article which had been forgotten about, but could have been useful in recent days. Or I have gotten excited about an article someone sent me, only to file it away somewhere where I had located the same article months before.
Currently in Apple's Panther, search is featured in all open Finder windows, in Mail, and in many other apps. What Spotlight will do will be to offer system-wide integrated search. To do that, it will use OS X's existing file system, and add new technologies that will allow searches for files, metadata, and indexed data.
It goes beyond employing the owner's keywords (and that's another lesson in itself which most people first are exposed to in Microsoft's Word app or Adobe Acrobat). Spotlight will also use importers to decipher and read a variety of metadata formats allowing extensibility for who-knows what the future of data indexing will bring. And Apple has released to developers kits to bring Spotlight features into their applications. Some will be excited by this, others won't see a future return to bother with such efforts.
It's possible that search and store app. developers who have been resistant to writing for the Mac platform might see a way to integrate their current Windows apps with Tiger's Spotlight to bring new enhanced features.
EverNote is one such application that has been threatening to develop its highly praised Windows app for the Mac, and who knows if Tiger will keep it away permanently or entice it to finally release a Mac product. It's a great irony that on its webpage, it features EverNote on a Treo 600 unit, as well as a laptop, except the laptop looks suspiciously like a 10 year old Apple Powerbook 180c! (See illustration at the top of the blog).
Here's the bottom line: Unless you have been living a Survivor-type lifestyle, more and more of the work you'll do, the communication you'll make with other people, the day to day matters of living such as bill paying, purchasing tickets, and just keeping track of your life (like reading or creating blogs) will be mediated via information technology systems, whether they're personal systems or within larger enterprises.
Just compare life now to that of just five years ago. How many monthly on-line subscriptions or payment schemes do you have now you didn't have before? $5 here, $10 there - it all adds up and even with direct debit systems, you still need to access the sites to be sure you haven't been overcharged, or your credit card account hasn't been exploited.
To go beyond merely keeping up with all the data smog, you'll need help you sort out what's important, from what could be important. And to titrate out what's redundant, you'll need clever search and storage technologies, processes and ideas. Anyone can bung a large hard drive into their computer and sharpen their retrieval by better archival methodologies which they have personalised. Most operating systems have some kind of built-in desktop search. Google will help you now, if you don't mind its limitations.
But I want to go deeper than that, to become truly smart and useful and tap into my own intuitive recall of data I have recorded. You and I will need better filtering and retrieval. In addition to reading blogs and sites which push to me information I trust (from past experience) to prove accurate and useful, the bottom line is I want my operating system to help out every little bit it can. I need to be able to trust it too. It's still comes back to a combination of people and technology.
How many organisations do you know who have an unofficial archivist/historian/librarian who always seems to have useful data at their fingertips? Look around and you'll start to notice them. Get friendly with them, and find a way to record what they tell you - both what you wanted to know, and extra tidbits that will come in handy who knows when? Trust and befriend them - they know much more than even they can tell you when you ask questions. Be warned though - they may work in places you least expect them to. You'll find them because your work colleagues will independently keep saying, "Just ask Harry... he knows". (Smart new CEOs and managers quickly locate the Harrys in their organisation and make sure they don't get fired in a downsize or promoted either for that matter.)
I have high hopes Spotlight will "do a Harry" on a personal level, then get out of my way (which is why I like Apple's software in general). And if it can do this, it'll be why Spotlight will be more important to the average user than first meets the eye.
If average users have begun to say farewell to spyware/malware/adware issues by considering the Mac, Spotlight could conceivably hasten those who are still caught in the herd mentality.
Not by their desire to escape what may have become a troublesome platform, but by being attracted to trustworthy and truly useful system-wide search applications. The irony is that these uses may occur in ways they don't even know about yet.
If the growing list of developers for Tiger comes up with something really special, let's hope Apple recognises the innovation of others and builds on it. Perhaps that Knowledge Navigator of Apple's fifteen years ago might get a little closer to reality.
Update - January 28: The Mac Observer (TMO) site of today carries a very interesting article on Apple's pursuit of a patent for Spotlight dating back to the year 2000, and which was granted a few days ago. It clearly lays out Spotlight's purposes and rationale, and the patent application date suggests Spotlight predates Microsoft Longhorn's WinFS search design, despite many in the IT industry pro-MS suggesting Apple copied MS.
You can see the TMO story here, and the full patent application here. |
|
|
|
