Big Data: Crowdsourced

In a previous blog post entitled Big Data: Big Opportunity, I talked about the potential for Going Non-Linear, by turning knowledge into Big Data apps in the cloud. However, the bottleneck here is Data Scientists: those special knowledge workers who can create the Big Data solutions to solve compelling business problems or generate valuable insights from large datasets. Now we have an answer to this bottleneck: it's a hot start-up called Kaggle - a crowdsourcing platform and ecosystem that harnesses the cognitive surplus of the world's Data Scientists.

As illustrated above, Kaggle enables enterprises and entrepreneurs to solve business problems through public and private competitions, where an ecosystem of over 25,000 (and growing) Data Scientists compete for the prize of winning a Predictive Modelling engagement. This typically makes use of the R tools from Revolution Analytics. In turn, Big Data cloud apps can then be built using a blend of a commercialised Apache Hadoop framework from players such as Cloudera or Hortonworks, delivered on a scalable cloud computing platform, such as Amazon Web Services (AWS).

Monetising Big Data cloud apps can leverage the crowdsourcing power of Kaggle with the insourcing power of Ciklum. This is where Data Scientists drawn from the Kaggle ecosystem can work with technical architects and software developers delivered through the Ciklum Own Software Development Team model. In practice, this means that the scarce resources of Data Scientists may be harnessed through the power of tapping into an ecosystem of thousands of subject matter experts. Underpinning this crowdsourcing on Kaggle is the ability to enable Big Data cloud app innovation.