Last week, I discussed data onboarding, and I received numerous questions about data profiling. It is a critical step in data processing. So today, let’s talk about data profiling – what it is, and some best practices for doing it right. Data profiling is the process of examining, analyzing, and summarizing data sets to understand their structure,
Data onboarding is a crucial process in any data-driven project. Doing it properly ensures data quality, consistency, and usability. Poor onboarding, on the other hand, can lead to flawed analyses and misguided decisions. Here are ten (10) dos and don’ts to guide you through the process… #1 – Do understand the data source. Before importing
A recurring concern I hear from heads of data science teams is that their data scientists spend too much time preparing their data for analysis rather doing actual analysis. As one hedge fund manager in London told me, “I don’t pay my quants to ‘fiddle’ with their data – I pay them to find alpha
I spend a lot of my time talking to the heads of data science teams about the projects they are working on and the data they need to complete their work successfully. Recently, I spoke with a manager who had just joined his firm and questioned how his team is currently organized. He felt like
