BIG data = a BIG problem

Yesterday I attended the 2nd LIKE Conference "From BIG data to little apps" I'd spoken at last years conference on social media and have been invited to take part in the LIKE microphone session where practitioners talked about how they were managing big data.

Before that though we were treated to three talks on what Big Data was and how it was being managed outside of the library/information sector. The first of these was from Dom Pollard of the Big Data Group Insight Group. Dom provided an excellent introduction to the concept of big data, including a good definition, which is that Big Data is about:

  • Volume, variety and velocity and Big Data is any amount of data that is too large to be stored, managed or analysed within a traditional database structure

However Dom stressed that it was important not to get hung up on Volume. Stressing that just because you have a lot of data doesn't necessarily mean it's Big Data. I think this is an important point. Whilst many organisations create a lot of content, take the average law firm and imagine how many emails they have in their document management system? it's going to be millions and millions, but is that Big Data? Probably not in the traditional sense, but it's certainly a lot of information that needs to be managed somehow.

Dom also talked about the benefits of collaboration with other Big Data sources arguing that:

  • Not all the data has to sit within your organisation
  • Creating mutually beneficial relationships can open up new possibilities
  • You should be prepared to embrace disruption and undergo a paradigm shift

The final key takeway was that you shouldn't blindly trust any data (including big data). Just because something is being presented to you in a nice format, doesn't mean it should be trusted. 

Our next presentation was from Michael Agar or Michael Agar Design. Michael talked briefly about how he created infographics to present complex information. One of Michael's first points was that all data needs a story. I think this is true to some extent, but like another attendee I suffer from infographic fatigue, in that all information nowadays seems to presented as an infographic. As I said in one of my tweets during the conference, some of these infographics just go on and on an on without really telling a story. I do believe that infographics can be an incredibly useful tool in presenting complex information, but they have to be used properly.

After Michael's presentation we then heard from Manny Cohen of the RM Group. Manny talked about how they had developed an app to present company information to consumers. You can read more about the app in this BBC Click video. Whilst this was interesting, it was unfortunately followed by a rather pointless exercise in looking at the future of information delivery, whilst the 15 minutes spent discussing this amongst ourselves was interesting, the 30 minutes spent analysing the results and then deciding on a winner was not. Especially as my table, which had probably the least tangible and well thought ideas won a glass of champagne each, which sadly I didn't get to enjoy.

The final session was the "LIKEmic" session where I was one of three presenters. During my session I talked about how we had been using Enterprise search to manage the data we were creating and present it to users. I also talked about how I believe intranets and specifically intranet dashboards have a key role to play in the delivery of complex information to users. As I said during my presentation, dashboards have the potential to make sense of complex information and tell users what they need to know rather then just present them with dead data.

Overall quite an interesting conference, although I didn't get as much out of it as last years conference on social media. I would also say I felt the venue last year was a bit better then the St Bride foundation. My main bugbear was that the room we were in was very large and the microphones didn't work particularly well.

[Photo credit - Big Data on Flickr]