Nodes¶
The easiest way to create a new node is create a class that inherits from CtrlNode
. You can place this class in one of the existing modules in mesmerize/pyqtgraphCore/flowchart/library
Become familiar with the Transmission object before creating a node. Almost all nodes work with a Transmission object for storing data. Make sure to conform to the conventions for naming of data columns and categorical columns.
Simple node¶
The simplest type of node performs an operation on a user-specified data column and doesn’t take any parameters.
Basic structure
class MyNode(CtrlNode):
"""Doc String that is shown when node is clicked on"""
nodeName = 'MyNode'
uiTemplate = <list of tuples, see below>
def processData(self, transmission: Transmission):
self.t = transmission.copy() #: input to this node
# .. do stuff to the Transmission DataFrame
params = <dict of analysis params>
# log the analysis that was done
self.t.history_trace.add_operation('all', 'mynode', params)
# Send the output
return self.t
Required class attributes:
nodeName: (str) The name prefix used when instances of this node are created in the flowchart
uiTemplate: (list) List of UI element tuples (see section)
If the node only has one input and one output terminal it is sufficient to create a processData method that performs the node’s analysis operation(s).
Example¶
1class Derivative(CtrlNode):
2"""Return the Derivative of a curve."""
3nodeName = 'Derivative'
4uiTemplate = [('data_column', 'combo', {}),
5 ('Apply', 'check', {'checked': False, 'applyBox': True})
6 ]
7
8# If there is only one input and one output temrinal, processData will
9# always have a single argument which is just the input transmission,
10# i.e. the output from the previous node.
11def processData(self, transmission: Transmission):
12 # the input transmission
13 self.t = transmission
14
15 # If a comboBox widget named 'data_column' is specified in the
16 # uiTemplate, you can update its contents using the following method.
17 # This will populate the comboBox with all the data columns from the
18 # input transmission and select the input data column as the
19 # output data column from the previous node.
20 self.set_data_column_combo_box()
21
22 # Check if the Apply checkbox is checked
23 if self.ctrls['Apply'].isChecked() is False:
24 return
25
26 # Make a copy of the input transmission so we can modify it to create an output
27 self.t = transmission.copy()
28
29 # By convention output columns are named after the node's name and in all caps
30 # Columns containing numerical data have a leading underscore
31 output_column = '_DERIVATIVE'
32
33 # Perform this nodees operation
34 self.t.df[output_column] = self.t.df[self.data_column].apply(np.gradient)
35
36 # Set tranmission's `last_output` attribute as the name of the output column
37 # This is used by the next node to know what thet last output data was
38 self.t.last_output = output_column
39
40 # Create a dict of parameters that this node used
41 # Usually a dict that captures the state of the uiTemplate
42 # the transmission `last_unit` attribute is the data units of the data
43 # in the output column (i.e. `t.last_output`). Change it only if the data units change
44 params = {'data_column': self.data_column,
45 'units': self.t.last_unit
46 }
47
48 # Add a log of this node's operation to the transmission's `HistoryTrace` instance
49 # Nodes usually perform an operation on all datablocks pass 'all' to the data_block_id argument
50 # By convention the operation name is the name of the node in lowercase letters
51 self.t.history_trace.add_operation(data_block_id='all', operation='derivative', parameters=params)
52
53 # return the modified transmission instance, which is then the output of this node
54 return self.t
Complex node¶
For a more complex node with multiple inputs and/or outputs you will need to explicitly specify the terminals when instantiating the parent CtrlNode
and create a simple override of the process() method.
Format of the dict specifying the node’s terminals:
{
<terminal name (str)>: {'io': <'in' or 'out'>},
<another terminal name (str)>: {'io', <'in' or 'out'>},
<another terminal name (str)>: {'io', <'in' or 'out'>}
...
}
Override the process() method simply pass all kwargs to a processData() method and return the output The processData() method must return a dict. This dict must have keys that correspond to the specified output terminals. The values of these keys are the outputs from the respective terminals.
Here is a trimmed down example from the LDA node
:
1class LDA(CtrlNode):
2"""Linear Discriminant Analysis, uses sklearn"""
3nodeName = "LDA"
4uiTemplate = [('train_data', 'list_widget', {'selection_mode': QtWidgets.QAbstractItemView.ExtendedSelection,
5 'toolTip': 'Column containing the training data'}),
6 ('train_labels', 'combo', {'toolTip': 'Column containing training labels'}),
7 ('solver', 'combo', {'items': ['svd', 'lsqr', 'eigen']}),
8 ('shrinkage', 'combo', {'items': ['None', 'auto', 'value']}),
9 ('shrinkage_val', 'doubleSpin', {'min': 0.0, 'max': 1.0, 'step': 0.1, 'value': 0.5}),
10 ('n_components', 'intSpin', {'min': 2, 'max': 1000, 'step': 1, 'value': 2}),
11 ('tol', 'intSpin', {'min': -50, 'max': 0, 'step': 1, 'value': -4}),
12 ('score', 'lineEdit', {}),
13 ('predict_on', 'list_widget', {'selection_mode': QtWidgets.QAbstractItemView.ExtendedSelection,
14 'toolTip': 'Data column of the input "predict" Transmission\n'
15 'that is used for predicting from the model'}),
16 ('Apply', 'check', {'applyBox': True, 'checked': False})
17 ]
18
19def __init__(self, name, **kwargs):
20 # Specify the terminals with a dict
21 CtrlNode.__init__(self, name, terminals={'train': {'io': 'in'},
22 'predict': {'io': 'in'},
23
24 'T': {'io': 'out'},
25 'coef': {'io': 'out'},
26 'means': {'io': 'out'},
27 'predicted': {'io': 'out'}
28 },
29 **kwargs)
30 self.ctrls['score'].setReadOnly(True)
31
32# Very simple override
33def process(self, **kwargs):
34 return self.processData(**kwargs)
35
36def processData(self, train: Transmission, predict: Transmission):
37 self.t = train.copy() #: Transmisison instance containing the training data with the labels
38 self.to_predict = predict.copy() #: Transmission instance containing the data to predict after fitting on the the training data
39
40 # function from mesmerize.analysis.utils
41 dcols, ccols, ucols = organize_dataframe_columns(self.t.df.columns)
42
43 # Set available options for training data & labels
44 self.ctrls['train_data'].setItems(dcols)
45 self.ctrls['train_labels'].setItems(ccols)
46
47 dcols = organize_dataframe_columns(self.to_predct.df.columns)
48 # Set available data column options for predicting on
49 self.ctrls['predict_on'].setItems(dcols)
50
51 # Process further only if Apply is checked
52 if not self.ctrls['Apply'].isChecked():
53 return
54
55 # Get the user-set parameters
56 train_column = self.ctrls['train_data'].currentText()
57
58 # ... get other params
59 n_components = self.ctrls['n_components'].value()
60
61 # ... do stuff
62
63 # This node ouputs separate transmissions that are all logged
64 self.t.history_trace.add_operation('all', 'lda', params)
65 self.t_coef.history_trace.add_operation('all', 'lda', params)
66 self.t_means.history_trace.add_operation('all', 'lda', params)
67
68 # the `to_predict` transmission is logged differently
69 self.to_predict.history_trace.add_operations('all', 'lda-predict', params_predict)
70
71 # dict for organizing this node's outputs
72 # The keys MUST be the same those specified for this node's output terminals
73 out = {'T': self.t,
74 'coef': self.t_coef,
75 'means': self.t_means,
76 'predicated': self.to_predct
77 }
78
79 return out
uiTemplate¶
Specify the uiTemplate attribute as a list of tuples.
One tuple per UI element with the following structure:
(<name (str)>, <type (str)>, <dict of attributes to set>)
Examples:
('dist_metric', 'combo', {'items': ['euclidean', 'wasserstein', 'bah'], 'toolTip': 'distance metric to use'})
('n_components', 'intSpin', {'min': 2, 'max': 10, 'value': 2, 'step': 1, 'toolTip': 'number of components'})
('data_columns', 'list_widget', {'selection_mode': QtWidgets.QAbstractItemView.ExtendedSelection})
The name can be anything. Accepted types and accepted attributes are outlined below
widget type |
attributes that can be set |
---|---|
intSpin |
min (int): minimum value allowed in the spinbox
max (int): maximum value allowed
step (int): step size
value (int): default value
|
doubleSpin |
min (float): minimum value allowed in the spinbox
max (float): maximum value allowed
step (float): step size
value (float): default value
|
check |
checked (bool): default state of the checkBox
applyBox (bool): Whether this is an “Apply checkbox”
|
radioBtn |
checked (bool): default state of this radioButton
|
combo |
items (list): default list of items that will be set in the comboBox
|
list_widget |
items (list): default list of items that will be set in the list_widget
selection_mode: One of the accepted QAbstractItemView selection modes
|
lineEdit |
text (str): default text in the line edit
placeHolder (str): placeholder text
readOnly (bool): set as read only
|
plainTextEdit |
text (str): default text in the text edit
placeHolder (str): placeholder text
|
label |
text (str): default text
|
button |
text (str): default text on the button
checkable (bool): whether this button is checkable
|
color |
Does not take any attributes |
All UI widget types outlined above take ‘toolTip’ as an attribute which can be used to display tooltips