Predictions for 2015
in 2015, Advanced Analytics, Business Intelligence, data management, Data Quality, data visualization, Predictions
As we get ready to kick off the new year, it's time to think about what we've seen and think about where we're going. Here are predictions about the market for BI and analytics, data integrity, and integration in 2015.
1. The Internet of Things will continue to gain momentum in 2015, but will still be on the early part of the curve.
Yes, eventually everything will be connected. I'm excited about it -- but that's part of the reason I hesitate to predict an immediate uptake. I believe that, in 2015, the dramatic increase in the number of connected devices will be related to high-value items: cars, trucks, airplanes, smart watches, and the like. Those types of items also produce a great deal of data, with multiple measures along the time, user, human-interaction, and other dimensions. This data will be captured and collected to show when an aircraft needs repairs, when an automobile's driver is exhibiting behavior that makes it appear that an unauthorized driver might be at the wheel, or when a truck driver might be overtired or drunk.
While we've seen the number of connected devices continue to rise, most people still aren't putting network keys into their toasters. Toasters will remain network-free for the most part in 2015, too, even as high-value items get connected. Expect toasters to get connected, eventually -- as people get readings of the nutritional value of the bread they toast, their energy consumption, and their carbon footprints, among other things -- but the more mundane items won't be connected for another year or two yet.
Even in these early days, the data collection will be vast, staggeringly more than IT shops are used to dealing with. Interestingly, it will become so vast that...
2. ...most people won't know what to do with all of the data they have. We already see people collecting far more data than they know what to do with. We already see people "hadumping" all sorts of data into Hadoop clusters without knowing how it's going to be used. They're treating data the way I treat the junk that ends up in my garage.
Ultimately, data has to be used in order for it to have value. And the usage of data isn't going to be constrained by technology per se, but by the curation process for all of the data -- structured, unstructured, semi-structured, big, small, individual records, big batches, you name it. Where we once placed value in the people who understood the data model, in 2015 we'll just start to place value in the people who know where the different pieces of it live. And that knowledge can't just live in a person's head, so Big Documentation is around the corner.
None of this means that the term Big Data will go away, by the way, although the term will continue to decline in relevance.
And speaking of decline...
3. Data warehouses will continue their decline, but not completely.
Data warehouses used to be a centerpiece of analytical architecture, but they're increasingly Just One More Data Source that gets used for analytics or operations. With an increased emphasis on Big Data stores, the structured data warehouse is becoming more difficult to justify.
That said, there's still a strong need in some areas for data that's clean, reconciled across multiple domains, and structured for rapid analytics. Sometimes the need for that will be satisfied through master data management; sometimes it will be satisfied through a data warehouse. The warehouse isn't going away, but it's going to be more important than ever to have a very strong business case to justify it.
Even as data warehouses get become more specialized, however, data movement will become more common through...
4. ...personal, business-user-oriented data manipulation tools.
Extraction, transformation, and load (ETL) tools used to be the purview of skilled technicians in IT. They pulled together data from many different sources, reconciled different data models, and staged data for large segments of the enterprise to use.
Modern analysis is becoming less like that. Business users are increasingly happy to pull together more of their own data for analysis. Frequently it's all from one or two systems, so keys are easier to reconcile; often there's a Big Data source in the mix, which means people are blending data based on what makes sense to them rather than strictly defined key relationships. They're also incorporating more spreadsheets and other personal data.
When I say "business users", of course, I mean "technology-savvy, data-oriented business users." Most business users don't have the skill, time, or talent to source their own data. For the vast majority of people, then, InfoApps are still a better way to go than just handing people a data manipulation tool. IT's job will shift somewhat, then, from providing full-blown data warehouses to providing tools to the tech-savvy on one hand, and apps to end users on the other to keep them from "getting into trouble with data".
Speaking of which...
5. ...I expect personal predictive technology to start to come to the fore in 2015 -- and fail. As analysts see that they can use data discovery and other analytical tools, they'll want the power of predictions to fall within their grasp, too. Unfortunately, most people don't understand statistics well enough (see the "Monty Hall problem") to make predictive models that really work, no matter how simple the tools make it. If anything, simple tools are more likely to get them into trouble.
We've been providing predictive technology in our WebFOCUS platform for years now, as part of the platform. It's still incredibly valuable there -- rookie police officers can receive predictions about where crime is likely to occur, helping them fight crime as well as 20-year veterans -- and deploying predictive analytics to the masses will be important for everyone who can do it. But the allure of predictive analytics will start to move to the business user, and that will become problematic where it happens. We'll continue to stand by the platform method to embed predictive analytics into InfoApps for wide-ranging deployment to non-technical users.
6. Some of the things that moved more slowly this year will pick up the pace. The bloom will continue to come off the rose for ungoverned data discovery, as more people hit the wall with individual tools. Analytical apps will continue to develop for smartphones, tablets, and the like for non-technical users. Data quality continues to gather more interest and respect as a business problem.
Having said all of that:
Many of these issues will require us to react as a company by helping people address their requirements. Data strategy is a huge focus. Redefining self-service analytics, to include InfoApps alongside data discovery, is another. Operationalizing business insights and monetizing the data that lives in Big Data stores is a third. There are more, but these are some of the key areas we believe we'll be able to help organizations move forward as their data foundations shift beneath them somewhat this year.
If you'd like, use the comments to tell us what you think!