Data infrastructure is maturing

As data becomes the key to businesses success, the infrastructure that supports it is maturing.

Nearly all companies have some sort of data infrastructure. This can start with a few excel spreadsheets held on laptops, and can grow to huge data centres managed my internet giants like Google. This infrastructure can serve two main purposes:

โ€œTo help business leaders make better decisions through the use of data (analytic use cases) and to build data intelligence into customer-facing applications, including via machine learning (operational use cases).โ€

Broadly, the analytics use case tends to be centred around a data warehouse, and the operational approach tends to focus on a data lake. This article gives three useful blueprints for data architectures, depending on your business requirements.:

  • Modern BI (cloud based data warehouse + visualisation tools - suitable for most companies)
  • Big Data (big enterprises or tech firms with complex needs. )
  • AI/ML (the least mature and most difficult to get right, many companies build their own. Lots of open source projects. Many donโ€™t need this.)

BEWARE: the article linked is written by A16Z, who have a significant investment in data infrastructure company Databricks, that happens to be at the centre of most of these diagrams. Despite this, itโ€™s a useful, if slightly biased, introduction.

๐Ÿ“– Read more here (2,382 words) ๐Ÿ“–