|
|
# What is multiscale ?
|
|
|
# What is Hystorian ?
|
|
|
|
|
|
Multiscale is a project born from our AFM group to handle multiple file format and converted all of them into hdf5.
|
|
|
Keeping track of the post-treatement done on our data can sometimes be cumbersome.
|
|
|
|
|
|
As so, all the processing can be done on the well defined hdf5 format, allowing for intercompatibility of the data analysis.
|
|
|
<img src="uploads/cfb65b1c76ddbf73fab6a79c16b24e0b/Capture.PNG" alt="drawing" width="500"/>
|
|
|
|
|
|
# Tutorials
|
|
|
Hystorian is a project born from our AFM group to solve this issue. It is a python package with two main functionalities:
|
|
|
|
|
|
1. It handles multiple file format and converted all of them into uniformized hdf5 files. <img src="uploads/f87976f1f287d14ccfd65503d7f27268/Capture.PNG" alt="drawing" width="150"/>
|
|
|
|
|
|
2. It keeps track of the processing done on the data. <img src="uploads/7ac15699feeb379123ae82d491502f9e/Capture.PNG" alt="drawing" width="250"/>
|
|
|
|
|
|
## Structure of an hdf5 file
|
|
|
An hdf5 file has a tree-like structure consisting of groups (folders) and datasets. Each groups and datasets can have attributes attached to them. An attribute is a small metadata containing informations about the corresponding dataset or group. Here we use them to store general information about the data or parameters of the post-processing.
|
|
|
|
|
|
The tree-structure we use in Hystorian is the following :
|
|
|
1. A dataset group, containing all the raw data you want to regroup inside a single hdf5.
|
|
|
2. A metadata
|
|
|
|
|
|
<img src="uploads/082057ad199de37ba34ed06b58aba87b/Capture.PNG" alt="drawing" width="350"/>
|
|
|
|
|
|
## m_apply
|
|
|
|
|
|
Our goal was to streamline the process and make the usage of Hystorian as seamless as possible for people already using Python. The key to this is the [`m_apply`](/Functions/Core/m_apply) function. It allows the application of any usual python function working on array on a datasets inside an hdf5 and to write the results into the same hdf5, while keeping all the key arguments of the function stored with the datasets inside the hdf5 file. Tutorials below show you how to use it in real case examples.
|
|
|
|
|
|
<img src="uploads/f26acd45d64310ba4dea78704f8f6b14/Capture.PNG" alt="drawing" width="450"/>
|
|
|
|
|
|
## Tutorials
|
|
|
|
|
|
A few tutorial were written by R.Bulanadi and can be found in the examples folder of the project, or in the links below:
|
|
|
|
... | ... | |