Prospective clients of DataSalon’s MasterVision service are sometimes nervous about the data supply side of the process – in most of the demos we do, somebody will say, ‘This looks great, but our data’s kind of messy…’ Fortunately, we’re data experts, with skills extending far beyond data analysis, and there’s a lot we can do to help our clients get their data into shape.
Data discovery
By studying your use cases for how you want to be able to search and analyse your data, we can advise on what data you need, as well as looking at your business processes and systems to establish where that data resides and how to retrieve it.
Data collection
We’re already using APIs to collect data automatically on behalf of our clients from Aries Editorial Manager, Clarivate ScholarOne, and Crossref, as well as employing the SUSHI standard to collect COUNTER usage data. We’re always happy to investigate other APIs and guide you if you’re struggling to retrieve data from a particular system.
Data design
If you’re setting up a data export from scratch, we can discuss with you which fields are needed to support your use cases. And where the only export option is a huge data dump, we can pull out the fields you actually need, in order to avoid information overload. We can also help you to pull in missing information, by inferring new fields from existing ones, or by using mapping files.
Data aggregation
If you need to combine data from multiple sources (for example if you’ve changed the system you use and want to include data from both the old and the new system), we can merge the data for you. We also have incremental update processes to deal with data from systems which produce monthly reports. And there are many other aggregation options, such as rolling data up to the best level to support your needs or setting up pivots to present it in a better way for analysis.
Data validation and cleansing
‘Garbage in, garbage out’ applies just as much to data analysis as it does to programming, so we always analyse data sources to identify potential problems – and we’ve developed lots of tools to address those issues. This isn’t always easy, but if there’s a pattern (and we have lots of experience of spotting those patterns), then we can usually come up with an automated fix. And in cases where manual intervention is preferable, we can report on what needs resolving.
Data compliance
We offer support in meeting legal and operational requirements, such as data protection laws and funder mandates on open access. For example, marketing permissions can be difficult to track across different business systems – we can help you formulate rules to make sense of conflicting opt-outs, and implement those rules for you.
These are the main data services we offer, but different clients will pick and choose the ones relevant to them, and we’re always happy to discuss knotty data issues – our technical team love a challenge! So if data processing difficulties are holding you back, why not get in touch to find out more?
The DataSalon Blog 