As data becomes the key to businesses success, the infrastructure that supports it is maturing.
Nearly all companies have some sort of data infrastructure. This can start with a few excel spreadsheets held on laptops, and can grow to huge data centres managed my internet giants like Google. This infrastructure can serve two main purposes:
“To help business leaders make better decisions through the use of data (analytic use cases) and to build data intelligence into customer-facing applications, including via machine learning (operational use cases).”
Broadly, the analytics use case tends to be centred around a data warehouse, and the operational approach tends to focus on a data lake. This article gives three useful blueprints for data architectures, depending on your business requirements.:
- Modern BI (cloud based data warehouse + visualisation tools - suitable for most companies)
- Big Data (big enterprises or tech firms with complex needs. )
- AI/ML (the least mature and most difficult to get right, many companies build their own. Lots of open source projects. Many don’t need this.)
BEWARE: the article linked is written by A16Z, who have a significant investment in data infrastructure company Databricks, that happens to be at the centre of most of these diagrams. Despite this, it’s a useful, if slightly biased, introduction.📖 Read more here (2,382 words) 📖