Data Quality can sometimes feel like the Holy Grail, with those of us who generate, process, or use data doing our best impersonations of Sean Connery from “Indiana Jones and the Last Crusade.” We dedicate so much time, treasure, blood, sweat, tears… a lot of tears… trying to ensure the highest quality data. But what
Last week, I discussed data onboarding, and I received numerous questions about data profiling. It is a critical step in data processing. So today, let’s talk about data profiling – what it is, and some best practices for doing it right. Data profiling is the process of examining, analyzing, and summarizing data sets to understand their structure,
Data onboarding is a crucial process in any data-driven project. Doing it properly ensures data quality, consistency, and usability. Poor onboarding, on the other hand, can lead to flawed analyses and misguided decisions. Here are ten (10) dos and don’ts to guide you through the process… #1 – Do understand the data source. Before importing
A recurring concern I hear from heads of data science teams is that their data scientists spend too much time preparing their data for analysis rather doing actual analysis. As one hedge fund manager in London told me, “I don’t pay my quants to ‘fiddle’ with their data – I pay them to find alpha
I spend a lot of my time talking to the heads of data science teams about the projects they are working on and the data they need to complete their work successfully. Recently, I spoke with a manager who had just joined his firm and questioned how his team is currently organized. He felt like
Data In Harmony (DIH) recently expanded its Level 3 Order Book Data offering to include equities and ETFs from Japan and Singapore. We now offer the most granular order book data from CBOE Japan, Japannext, and Singapore Exchange. This compliments the existing Tokyo Stock Exchange data we already offer, along with the equities, ETFs, and
The Financial Conduct Authority (FCA), which regulates markets in the UK, sent out a press release hinting it might change rules concerning market data after it found that competition is “not working as well as it should”. As a result, the FCA wants to introduce a “consolidated tape” that would collect price data across the
Financial data users have been frustrated for years by the restrictions legacy data providers have put in their contracts to prohibit the sharing of data within an organization. The advent of cloud computing and remote working have exasperated the situation even more. It is unrealistic for data providers to expect all of the end-users of
Data In Harmony (DIH) recently expanded its Investment Funds data offering to include NAVs, reference data, and corporate actions for Unit Trusts and Open-Ended Investment Companies (OEIC) in the United Kingdom and Open-Ended Funds in Ireland. Adding to its USA Mutual Funds and global Exchange-Traded Funds (ETFs) data, DIH has added NAVs, reference data, and
Are you paying 5X more than your competitors (with your AUM) for the same exact data? Mike Carrodus and his colleagues at Substantive Research investigated what buy-side firms pay for the same market data. Their “apples-to-apples” comparisons reveal legacy data providers charge widely different rates for the same data. These pricing inconsistencies are often enormous.
