Data types used for analysis¶
Transmission¶
Inherits from BaseTransmission
-
class
mesmerize.
Transmission
(df: pandas.core.frame.DataFrame, history_trace: mesmerize.analysis.data_types.HistoryTrace, proj_path: Optional[str] = None, last_output: Optional[str] = None, last_unit: Optional[str] = None, ROI_DEFS: Optional[list] = None, STIM_DEFS: Optional[list] = None, CUSTOM_COLUMNS: Optional[list] = None, plot_state: Optional[dict] = None)[source]¶ The transmission object used throughout the flowchart
-
__init__
(df: pandas.core.frame.DataFrame, history_trace: mesmerize.analysis.data_types.HistoryTrace, proj_path: Optional[str] = None, last_output: Optional[str] = None, last_unit: Optional[str] = None, ROI_DEFS: Optional[list] = None, STIM_DEFS: Optional[list] = None, CUSTOM_COLUMNS: Optional[list] = None, plot_state: Optional[dict] = None)¶ Base class for common Transmission functions
- Parameters
df (pd.DataFrame) – Transmission dataframe
history_trace (HistoryTrace) – HistoryTrace object, keeps track of the nodes & node parameters the transmission has been processed through
proj_path (str) – Project path, necessary for the datapoint tracer
last_output (str) – Last data column that was appended via a node’s operation
last_unit (str) – Current units of the data. Refers to the units of column in last_output
plot_state (dict) – State of a plot, such as data and label columns. Used when saving interactive plots.
- Variables
df – DataFrame instance
last_output – Name of the DataFrame column that contains data from the most recent node
last_unit – The data units for the data in the column of ‘last_output’
plot_state – State of a plot, containing user entered plot parameters. Used for storing interactive plot states.
-
static
empty_df
(transmission, addCols: Optional[list] = None) → pandas.core.frame.DataFrame¶ Just a helper method to return an empty DataFrame with the same columns
- Parameters
transmission – Transmission object that forms the basis
addCols – list of columns to add
- Returns
The input transmission with an empty dataframe containing the same columns and any additional columns that were passed
-
classmethod
from_pickle
(path)¶ Load Transmission from a pickle.
- Parameters
path – file path, usually ends in .trn
-
to_pickle
(path: str)¶ Save Transmission as a pickle. Not recommended for sharing data, use
to_hdf5()
- Parameters
path – file path, usually ends in .trn
-
classmethod
from_hdf5
(path: str)¶ Create Transmission from an hdf5 file. See
HdfTools
for information on the file structure.- Parameters
path – file path, usually ends in .trn (.ptrn for plots)
-
to_hdf5
(path: str)¶ Save as an hdf5 file. Uses pytables to save the DataFrame, serielizes the HistoryTrace using JSON. See
HdfTools
- Parameters
path – file path, usually ends in .trn
-
get_proj_path
() → str¶ Get the project root dir associated to this Transmission.
- Returns
Root directory of the project
-
set_proj_path
(path: str)¶ Set the project root dir for this transmission.
Used for finding associated project files, for example the Datapoint Tracer uses it to find max and std projections of image sequences.
- Parameters
path – Root directory of the project
-
classmethod
from_proj
(proj_path: str, dataframe: pandas.core.frame.DataFrame, sub_dataframe_name: str = 'root', dataframe_filter_history: Optional[dict] = None)[source]¶ - Parameters
proj_path – root directory of the project
dataframe – Chosen Child DataFrame from the Mesmerize Project
sub_dataframe_name – Name of the sub DataFrame to load
dataframe_filter_history – Filter history of the child dataframe
-
static
_load_files
(proj_path: str, row: pandas.core.series.Series) → pandas.core.series.Series[source]¶ Loads npz of curve data and pickle files containing metadata using the paths specified in each row of the chosen sub-dataframe of the project
-
classmethod
merge
(transmissions: list)[source]¶ Merges a list of Transmissions into one transmission. A single DataFrame is created by simple concatenation. HistoryTrace objects are also merged using HistoryTrace.merge.
- Parameters
transmissions – A list containing Transmission objects to merge
- Returns
Merged transmission
-
BaseTransmission¶
-
class
mesmerize.analysis.data_types.
BaseTransmission
(df: pandas.core.frame.DataFrame, history_trace: mesmerize.analysis.data_types.HistoryTrace, proj_path: Optional[str] = None, last_output: Optional[str] = None, last_unit: Optional[str] = None, ROI_DEFS: Optional[list] = None, STIM_DEFS: Optional[list] = None, CUSTOM_COLUMNS: Optional[list] = None, plot_state: Optional[dict] = None)[source]¶ -
__init__
(df: pandas.core.frame.DataFrame, history_trace: mesmerize.analysis.data_types.HistoryTrace, proj_path: Optional[str] = None, last_output: Optional[str] = None, last_unit: Optional[str] = None, ROI_DEFS: Optional[list] = None, STIM_DEFS: Optional[list] = None, CUSTOM_COLUMNS: Optional[list] = None, plot_state: Optional[dict] = None)[source]¶ Base class for common Transmission functions
- Parameters
df (pd.DataFrame) – Transmission dataframe
history_trace (HistoryTrace) – HistoryTrace object, keeps track of the nodes & node parameters the transmission has been processed through
proj_path (str) – Project path, necessary for the datapoint tracer
last_output (str) – Last data column that was appended via a node’s operation
last_unit (str) – Current units of the data. Refers to the units of column in last_output
plot_state (dict) – State of a plot, such as data and label columns. Used when saving interactive plots.
- Variables
df – DataFrame instance
last_output – Name of the DataFrame column that contains data from the most recent node
last_unit – The data units for the data in the column of ‘last_output’
plot_state – State of a plot, containing user entered plot parameters. Used for storing interactive plot states.
-
plot_state
¶ If used in a plot, dict containing information about the plot state
-
to_hdf5
(path: str)[source]¶ Save as an hdf5 file. Uses pytables to save the DataFrame, serielizes the HistoryTrace using JSON. See
HdfTools
- Parameters
path – file path, usually ends in .trn
-
classmethod
from_hdf5
(path: str)[source]¶ Create Transmission from an hdf5 file. See
HdfTools
for information on the file structure.- Parameters
path – file path, usually ends in .trn (.ptrn for plots)
-
classmethod
from_pickle
(path)[source]¶ Load Transmission from a pickle.
- Parameters
path – file path, usually ends in .trn
-
to_pickle
(path: str)[source]¶ Save Transmission as a pickle. Not recommended for sharing data, use
to_hdf5()
- Parameters
path – file path, usually ends in .trn
-
static
empty_df
(transmission, addCols: Optional[list] = None) → pandas.core.frame.DataFrame[source]¶ Just a helper method to return an empty DataFrame with the same columns
- Parameters
transmission – Transmission object that forms the basis
addCols – list of columns to add
- Returns
The input transmission with an empty dataframe containing the same columns and any additional columns that were passed
-
get_proj_path
() → str[source]¶ Get the project root dir associated to this Transmission.
- Returns
Root directory of the project
-
HistoryTrace¶
-
class
mesmerize.analysis.data_types.
HistoryTrace
(history: Optional[Dict[Union[uuid.UUID, str], List[Dict]]] = None, data_blocks: Optional[List[Union[uuid.UUID, str]]] = None)[source]¶ Structure of a history trace:
A dict with keys that are the block_ids. Each dict value is a list of operation_dicts. Each operation_dict has a single key which is the name of the operation and the value of that key is the operation parameters.
{block_id_1: [
{operation_1:
{
param_1: a,
param_2: b,
param_n, z
}
},
{operation_2:
{
param_1: a,
param_n, z
}
},
…
{operation_n:
{
param_n: x
}
}
]
block_id_2: <list of operation dicts>,
…
block_id_n: <list of operation dicts>
}
The main dict illustrated above should never be worked with directly.
You must use the helper methods of this class to query or add information
-
__init__
(history: Optional[Dict[Union[uuid.UUID, str], List[Dict]]] = None, data_blocks: Optional[List[Union[uuid.UUID, str]]] = None)[source]¶ - Parameters
history – Dict containing a data block UUIDs as keys. The values are a list of dicts containing operation parameters.
data_blocks – List of data block UUIDs
- Variables
_history – The dict of the actual data, as illustrated above. Should not be accessed directly. Use the
history
property or call get_all_data_blocks_history()._data_blocks – List of all data blocks. Should not be accessed directly, use the
data_blocks
property instead.
-
property
data_blocks
¶ List of UUIDs that allow you to pin down the history of specific rows of the dataframe to their history as stored in the history trace data structure (self.history)
-
property
history
¶ The analysis log that is stored in the structure outlined in the doc string
-
create_data_block
(dataframe: pandas.core.frame.DataFrame) → Tuple[pandas.core.frame.DataFrame, uuid.UUID][source]¶ Creates a new UUID, assigns it to the input dataframe by setting the UUID in the _BLOCK_ column
- Parameters
dataframe – Assigns a block ID to this entire DataFrame.
-
_add_data_block
(data_block_id: uuid.UUID)[source]¶ Adds new datablock UUID to the list of datablocks in this instance. Throws exception if UUID already exists.
-
add_operation
(data_block_id: Union[uuid.UUID, str], operation: str, parameters: dict)[source]¶ Add a single operation, that is usually performed by a node, to the history trace. Added to all or specific datablock(s), depending on which datablock(s) the node performed the operation on
- Parameters
data_block_id – data_block_id to log the operation on to. either a UUID or ‘all’ to append the operation to all data blocks
operation – name of the operation, usually the same as the name of the node in all lowercase
parameters – operation parameters.
-
get_data_block_history
(data_block_id: Union[str, uuid.UUID], copy: bool = False) → List[dict][source]¶ Get the full history trace of a single data block.
Use copy=False if you want to modify the history trace of the data block.
-
get_operations_list
(data_block_id: Union[uuid.UUID, str]) → list[source]¶ Returns just a simple list of operations in the order that they were performed on the given datablock. To get the operations along with their parameters call get_data_block_history()
-
get_operation_params
(data_block_id: Union[uuid.UUID, str], operation: str) → dict[source]¶ Get the parameters dict for a specific operation that was performed on a specific data block
-
check_operation_exists
(data_block_id: uuid.UUID, operation: str) → bool[source]¶ Check if a specific operation was performed on a specific datablock
-
static
_to_uuid
(u: Union[str, uuid.UUID]) → uuid.UUID[source]¶ If argument ‘u’ is type <str> that can be formatted as a UUID, return it as UUID type. If argument ‘u’ is a UUID, just return it.
-
to_dict
() → dict[source]¶ Package the HistoryTrace instance as a dict. Converts all UUIDs to <str> representation for JSON compatibility.
-
static
from_dict
(d: dict) → dict[source]¶ Format a dict stored using HistoryTrace.to_dict so that it can be used to create a HistoryTrace instance. Converts all the <str> representations of UUID back to
<uuid.UUID>
types.- Parameters
d – dict containing appropriate ‘history’ and ‘datablocks’ keys. Must be packaged by HistoryTrace.to_dict()
- Returns
dict formatted so that it can be used to instantiate a HistoryTrace instance recapitulating the HistoryTrace it was packaged from.
-
to_json
(path: str)[source]¶ Save HistoryTrace to a JSON file.
- Parameters
path – file path, usually ends with .json
-
classmethod
from_json
(path: str)[source]¶ Instantiate HistoryTrace from JSON file (that was saved using HistoryTrace.to_json)
- Parameters
path – file path, usually ends with .json
-
classmethod
from_pickle
(path: str)[source]¶ Load HistoryTrace that was pickled
- Parameters
path – file path
-
classmethod
merge
(history_traces: list)[source]¶ Merge a list of HistoryTrace instances into one HistoryTrace instance. Useful when merging Transmission objects.
- Parameters
history_traces – list of HistoryTrace instances
-
draw_graph
(data_block_id: Union[str, uuid.UUID], **kwargs) → str[source]¶ Draw graph of a data block. kwargs are passed to mesmerize.plotting.utils.draw_graph
-
static
clean_history_trace
(db_history: list) → list[source]¶ Cleans up excessive data such as frequencies linspaces and linkage matrices so the graph is viewable.
- Parameters
db_history – data block history
- Returns
data block history with excessive params removed
-
__weakref__
¶ list of weak references to the object (if defined)
-