Data systems design

Opinions

5 min. read

A lot of companies are drowning in data

Where lack of data used to be the issue, we increasingly see companies who have too much data

From data scarcity to data overflow

Looking back at some of our recent projects, there is something that firmly stands out: a lot of companies are nearly drowning in data. Perhaps the truism that 'Data is the new gold', has nudged companies to collect data without a proper plan on how to use it. As a result we see companies struggling with data that is scattered, unstructured, unreliable etc; what we like to summarise as 'rogue data'. Rogue data can be a huge problem because it is not useful, but it is there, clogging up your systems, clouding understanding, and hampering what it was supposed to do: enhancing interoperability, acquiring better insights, and facilitating operational efficiency. Rogue data needs to be addressed. The answer might be found in Data Systems design.

We need people in there from the humanside, to make sure data system designs fit with human needs

User experience as a discipline will need to step up to system level abstractions

Data Systems design

Data system design refers to the process of creating a well-structured and efficient framework for managing and organizing data within an organization or system. It involves the development and implementation of appropriate strategies, methodologies, and architectures to ensure that data can be collected, stored, accessed, and analyzed effectively (definition from alooba) For now, this is primarily the domain of people who already have a strong occupational link with the 'data side', like IT architects, and data scientists. That's great, but we also need some people in there from the 'human side', to make sure that these data system design fit in well with human needs, especially for people who do not have data literacy as one of their superpowers. Succesful cooperation between the binary thinkers of the data world, and analogue thinkers of the human world, can be challenging, but we strongly feel it is a necessity, and we have proven that's it is feasible. To go far, we must go together.