Manu - BigData

big cloud data

Sun 02 February 2014

[caption id="" align="alignright" width="350"] Big cloud not computing (Photo credit: Wikipedia)[/caption]

This post should be untitled From cloud computing to big data to fast data.

The previously next big stuff: cloud

computing

[caption id="" align="alignright" width="75"] Cloud computing (Photo credit: Wikipedia)[/caption]

Once upon a time, a long time ago everyone spoke of cloud computing. No one was able to give a clear definition. Anytime someone explained it to me I thought "Ho! but it was called IaaS, or PaaS, or SaaS", or "From the end user point of view, is there a difference with minitel (centralised service running far away from your device)? From the service provider's point of view, is there a difference with virtualisation?"

I think, there is none. Only some marchandising stuff added. Maybe more theorical work has been made.

The next big stuff: big data

[caption id="" align="alignright" width="350"] (Not) Big (SQL) Data (Photo credit: Wikipedia)[/caption]

More and more often mathematics are the key element of a newly sold tool. The big buzz word is now big data, in machine learning world, the trend is deep (learning), company specialized in math tool engineering are acquired by multinational groups,...

Big data exists for more than a century to designate data not easily processed by the state-of-the-art technologies. The evolution we witness today is due to computing progresses (Moore's law and other equivalent theory), including automatic data creation [1. e.g. wi-fi/hybrid localisation and mobile phone GPS tracking] [2. e.g. automatic cartography]. It is not new (see yahoo labs publications) but anyone (at least at my work) want to include this 2 words to claim ressources, as they included cloud computing few years ago. As I never succeed in finding a good definition of cloud computing, let's see what I can do with math-related words:

The first maths tools were created to manage the number of animals in livestock. Maths evolve to a such abstraction level that only specialists[3. so called mathematician, or with the more catch-all word scientists] understand maths. The paradox is that anyone in everyday life uses maths or tool made thanks to maths [4. non-exhaustive list I think of:

gears in cars (straingthforward calculus links to engine power and speed)
prepaid stuff (cryptographical proof)
bitcoin usage (block chain and DHT)
wireless communication (signal processing)
weather forecasting (famous learning machine and inference process)
counting number of animals in livestock (simplest example)
recommendation systems

]. The fact is that maths is not easy to learn, not easy to see, not easy to understand, and is more and more important in every day world. Another fact is that companies work with obscur and really specialised maths'incantations[5. or it seems to be (deep learning, extreme learning machines, inference, ...)] for a long time without many financial benefits, including weather forecasting, electrical grid comsuption forecasting, and more financial-related work such as exchange values forecasting. Today computing companies are acquiring math-related skills (by recruiting specialistst or by acquiring companies).

There is still no clear definition of big data. I can only list some key word related to big data. When writing down any one of those on a peper, you could put big data in the title:

hadoop
mongo DB
noSQL
any figure bigger than lets say 1 million
high dimensional feature
sparse
real time / online alanysis
data mining / machine learning
grid / distributed computing

I think, the future of computer science is for people understanding maths, and able to include maths'tools into computer-based services. In other words, the future is for developpers with not only a huge ability to devellop software [6. with all that is needed, i.e. multithread, memory management, network communication interface, packer, GUI, ... ] but also with a solid math background [5. understanding previously unseen maths tools and capable to see the implementaiton problems such as floating point errors, database implementation,...]. Yes, looks like me. Maybe my vision is tighten by my working experience.

(Non-)Related articles