Principles
Preventing loss and corruption
Files can be lost accidentally in many different ways. Even if they are not lost completely, they can occasionally become corrupted. If a file is severely corrupted it may be unusable, but even subtle corruption may introduce errors which go unnoticed while affecting the outcome of your research.
- Regular backups: (ideally automated) to several different locations will ensure that if one copy is lost or corrupt, you can easily get it back. When deciding how often to back up, think about the maximum number of days' work you would be prepared to lose. This is why we would recommend using Sharepoint for your project data.
Data is stored on the University of Stirling's instance of Microsoft Office 365. using either Sharepoint or Teams. They are both Tier D-compliant. This includes the following standards: ISO 27001, ISO 27018, SSAE16 SOC 1 and SOC 2, HIPAA, and EU Model Clauses (EUMC)."This is a cloud storage solution: Sharepoint and Teams enforce team-wide and organization-wide two-factor authentication, single sign-on through Active Directory, and encryption of data in transit and at rest. Files are stored in SharePoint and are backed by SharePoint encryption. If a team member accidentally deletes data it can be retrieved by the project owner and or the IT administrator
- Non-digital data: If you have data which is not kept on computer, you should make sure that is protected too.
Controlling access
In many cases, you may wish to restrict access to your data to a specific list of individuals. This might be because it is commercially sensitive to you or an industrial partner, or includes sensitive personal information covered by GDPR or they are your collaborators.
This is possible by adding members to your site - see how to
Research data should not be held exclusively on any local storage media, e.g., a pen drive as these can be easily misplaced
- Legal requirements: You may be under legal and/or contractual obligations to protect your data. If you're not sure, you can discuss this with the university's legal office, who can give you advice on your collaboration or consortium agreements and regulations such as GDPR.
- Use of secure systems: One way to restrict access is to use a password-protected system, as above. Commercial services such as Dropbox may be convenient, but are unlikely to provide sufficient protection against unauthorised access.
- Secure passwords: Passwords are often the weak link in any secure system. Make sure you choose passwords that are long and difficult to guess. Writing them down is OK, as long as you protect your written-down password very well, just like you would with your house or car keys.
- Encryption: The University of Stirling advises that all research data, especially confidential, is stored and transmitted in an encrypted format. All new university owned laptops will now be encrypted as standard. Data stored on transportable devices such as memory sticks or laptops should be encrypted at all times to prevent the loss of data due to loss or theft of the device. Care should be taken to ensure that encryption keys are not stored with the data they relate to and only known to essential members of the research team. You will sometimes need to send data to people who don't have access to your secure storage system. Encrypting a file before you send it via insecure means (e.g. email) ensures that the contents can only be read by someone who has the key.
File management and formats
There are many 'right' ways of organising your files so think about what makes sense for your research. You also need to ensure that you keep enough information to interpret the data.
If for example you're doing experimental work, you might want to organise the results into folders by the date you conducted the experiment or by a key experimental condition. For a small number of files, longer, more descriptive titles can be useful. (But don't go crazy: try to keep the whole name on the screen).
However you choose to arrange your files, make sure you write down what you've decided in an index file. (A Word or text document is great for this index file, that you keep along side the other files). This only takes a few minutes, but can save hours of searching later. You many need to update your index file as your research develops. Edinburgh University have produced a guide on naming electronic files.
Describing data
If it's never happened to you, you'd be amazed how quickly data becomes unusable because key details of the context have been forgotten. Whatever you need to make sense of your data, you should keep this with the data files themselves. This could be reactant concentrations and temperatures, details of how a sample was chosen.
For lab-based research, this is often recorded in the lab notebook, so make sure this is kept safe. Record the lab notebook page number with the data files, and if possible, scan the page(s) in and keep them with the data too. Even if you're not based in a lab, you may have a research journal that you can use in the same way.
This information also helps when deciding ownership and assigning credit, so make sure you keep a note of who collected the data and when, especially if it's not you. If it's relevant for your research, you should also keep safe copies of any legal documentation, such as consent forms or COSSH forms. Your research group, lab or department may have an existing filing system for this, so ask your supervisor.
Introduction to Metadata, with a festive theme (Scientific American)
Documenting your data (UK Data Archive)
Documentation and Metadata (MIT Libraries)
Metadata for Data Management (University of North Carolina at Chapel Hill)
Format your data
Research data exist in many different forms: textual, numerical, databases, geospatial, images, audio-visual recordings, and data generated by machines or instruments. Digital data exists in specific file formats which are coded so that a software programme can read and interpret these data.
Using standard and interchangeable or open lossless data formats ensures longer-term usability of data. For longterm preservation, digital data is converted to such formats.
Data files should be clearly named, well organised, structured and quality and version controlled throughout the research. It is vital to develop suitable procedures before data gathering starts in order to adhere to any conventions, instructions, guidelines or templates that will help to ensure quality and consistency across a data collection.
See the UK Data Service for more information.
Handling non-digital data
Not all research data is digital. Most researchers keep hand-written laboratory notebooks, journals and other materials which are not kept on a computer at all.
Paper Files
Anything stored on paper can be scanned fairly easily: find out how to scan directly to your University storage using any of the managed printers on campus (Library). See for more information on how to scan. If it's not easy to scan, you could try taking a digital photo, but check the quality of the image to make sure you can use it if you lose the original.
Audio Files
Audio recordings can easily be turned into digital sound files, if the sound content is important, or transcribed if only the words are needed. You can do this yourself, or employ a professional transcription service if you have a lot of recordings to digitise. See the UK Data Service for further advice on Transcription
Other Files
Other materials can also be digitised, with varying degrees of difficulty. If the data or artefact absolutely cannot be digitised for some reason, you should make sure that it's protected some other way. A fireproof safe could be a good investment. Find out more about how to digitalise your data via the UK Data Service.