Training to Climb an Everest of Digital Data
It’s not just technology companies and the federal government that have these challenges. Every mid/large enterprise is drowning in data. Unfortunately, the approach most often taken by IT organizations is to view data as a liability and delete it as soon as possible. This is the case even though much of the data is proprietary, meaning it is data competitors don’t have. Some of the search-based technologies I am writing about this quarter can be used to address this problem. But more research is still needed.
It is a rare criticism of elite American university students that they do not think big enough. But that is exactly the complaint from some of the largest technology companies and the federal government.
It was not long ago that the notion of one company having anything close to 40 billion photos would have seemed tough to fathom. Google, meanwhile, churns through 20 times that amount of information every single day just running data analysis jobs. In short order, DNA sequencing systems too will generate many petabytes of information a year.