Table of Contents¶
In [1]:
# first we define relevant directories
import sys
# in case PySeqLab package is not installed,
# we can download the package repository from https://bitbucket.org/A_2/pyseqlab
# and then we add the location of the repository to the python system path
# location of the PySeqLab repository on disk -- INSERT location or discard if PySeqLab package is already installed
pyseqlab_package_dir = ""
sys.path.insert(0, pyseqlab_package_dir)
import os
# project directory
project_dir = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
# src directory of the project
src_dir = os.path.join(project_dir, 'src')
sys.path.insert(0, src_dir)
# get the tutorials dir
tutorials_dir = os.path.join(project_dir, 'tutorials')
# to use for customizing the display/format of the cells
from IPython.core.display import HTML
with open(os.path.join(tutorials_dir, 'pseqlab_base.css')) as f:
css = "".join(f.readlines())
HTML(css)
Out[1]:
1. Objectives and goals¶
In this tutorial, we will learn about:
- the process/workflow for building part-of-speech tagger (i.e. based on CRFs model formalism)
- training the built model and evaluating its performance
- reviving the trained model and decoding new sequences (i.e. test sequences that were not used for training)
1.1 Problem statement¶
Given a set of sentences where each sentence is composed of words, the goal is to predict the part-of-speech label of every word in the sentence.
The goal of the GENIA POS annotation task is to build part-of-speech taggers using medical-related texts/corpus.
Reminder: To work with this tutorial interactively, we need to clone the part-of-speech tagger repository from bitbucket to our disk locally. Then, navigate to [cloned_package_dir]/tutorials where [cloned_package_dir] is the path to the cloned package folder.
NB: PySeqLab should be already installed or included in the python system path before we proceed.
2. Training part-of-speech tagger¶
The src directory in the cloned repository includes the following main modules:
genia_xml_parser.pypos_attr_extractor.pytrain_tagger_workflow.py
As a prerequisite, refer to these tutorials describing in detail the model building and training process using PySeqLab package.
2.1 Dataset¶
To build our training dataset, we processed the raw files using genia_xml_parser.py module. This process generated our final training dataset (GENIA_dataset_v1.txt). The tree path for the dataset directory should look like this:
├── dataset │ ├── GENIA_POS │ │ ├── GENIAcorpus3.02.pos.txt │ │ ├── GENIAcorpus3.02.merged.xml │ │ ├── LICENSE │ │ ├── GENIA_dataset_v1.txt │ │ ├── genia_pos_xml_structure │ │ ├── GENIAcorpus3.02.pos.xml │ │ ├── gpml.merged.dtd
In our current setting, we will randomly split our dataset (i.e. GENIA_dataset_v1.txt) into 90% for training and 10% for testing similar to the approach reported in (Tsuruoka et al., 2005).
2.2 Attributes and features extraction¶
We start by defining our attribute extractor that will be used to generate attributes from the parsed sequences. Our attribute extractor POS_AttributeExtractor is subclass of GenericAttributeExtractor class implemented in pos_attr_extractor.py module. It defines attributes based on the words track in addition to new computed attributes that measure/capture orthographic, shape and prefix/suffix attributes at each position in the sequence. Below is an example of the attributes extracted using our POS_AttributeExtractor class from a sequence in our training file.
After defining our attribute extractor, we define the feature templates that are used by the feature extractors to generate features. Feature templates and feature extraction are described in detail in this tutorial.
In the train_tagger_workflow.py module, we define our feature templates using template_config() function.
def template_config():
template_generator = TemplateGenerator()
template_XY = {}
# generating template for attr_name = w
template_generator.generate_template_XY('w',
('1-gram:2-gram:3-gram', range(-2,3)),
'1-state:2-states:3-states',
template_XY)
# generating template for the prefix and suffix
attr_names = ('prefix', 'suffix')
for attr_name in attr_names:
for i in range(2, 6):
template_generator.generate_template_XY('{}_{}'.format(attr_name, i),
('1-gram', range(0,1)),
'1-state',
template_XY)
attr_names = ('hasdash', 'hasnum', 'hascap', 'hasallcaps')
for attr_name in attr_names:
template_generator.generate_template_XY(attr_name,
('1-gram', range(0,1)),
'1-state',
template_XY)
attr_names = ('shape', )
for attr_name in attr_names:
template_generator.generate_template_XY(attr_name,
('1-gram:2-gram:3-gram', range(-1,2)),
'1-state',
template_XY)
template_Y = {'Y':()}
return(template_XY, template_Y)
The defined templates include multiple tracks (i.e. w, prefix_2:5, suffix_2:5, hasdash, hasnum, hascap, hasallcaps, and shape) representing the words and the computed attributes based on the words track. The notation prefix_2:5 represents multiple attributes (i.e. newly computed tracks) based on varying prefix lengths such as prefixes of length 2, 3, 4, and 5.
For the words track (i.e. w):
- We define a window of size 5 centered at each position in the sequence. We pass through the sequence from left to right, where at each position, we construct a window of size 5 (a window that includes attributes at two previous positions, current position, and two forward/future positions)
- We extract 1 to 3 grams (i.e.
1-gram:2-gram:3-gram) in the specified window - We join these attributes with (1) the current state, (2) the previous and current state, and (3) the two previous states and the current state (i.e.
Y labels)
For the following tracks {prefix_2:5, suffix_2:5, hasdash, hasnum, hascap, hasallcaps}:
- We define a window of size 1 centered at each position in the sequence. We pass through the sequence from left to right, where at each position, we construct a window of size 1 (a window that includes attribute only at the current position)
- We extract unigrams (i.e.
1-gram) in the specified window - We join these attributes with (1) the current state (i.e.
Y labels)
For the shape track:
- We define a window of size 3 centered at each position in the sequence. We pass through the sequence from left to right, where at each position, we construct a window of size 3 (a window that includes attributes at one previous position, current position and one future position)
- We extract unigrams, bigrams and trigrams (i.e.
1-gram:2-gram:3-gram) in the specified window - We join these attributes with (1) the current state (i.e.
Y labels)
2.3 Models and optimization options¶
In the train_tagger_workflow.py module, we implement the training workflow. In this section, we describe the training setup and the chosen options for performing the training.
We used the following classes:
- feature extractor (
HOFeatureExtractor), - CRFs model (
HOCRFAD) and - CRFs model representation (
HOCRFADModelRepresentation)
For the training method (i.e. optimization options), we used the following options:
- structured perceptron (
method = COLLINS-PERCEPTRON), - with beam search enabled (
beam_size = 5) and update type equal to max (update_type = max-fast), - and 10 passes through the training data (
num_epochs = 10)
To run the training process, we use run_training(optimization_options, template_config) function. We pass the optimization options and the function generating the defined feature templates (see this code snippet). The training process will perform the following:
- read the training file (i.e.
GENIA_dataset_v1.txt) and parse it into sequences - process and dump the parsed sequences on disk in a relevant format for the learning framework
- define a data split in which we randomly take 90% of the sequences for training and 10% for testing
- build a model based on the processed training sequences (i.e. 90% share)
- train the model weights (i.e. estimate the feature weights) using the specified optimization method
- use the trained model to decode the training sequences (i.e. 90% share) and write the result to a file
- use the trained model to decode the testing sequences (i.e. 10% share) and write the result to a file
- return the path to the trained model directory
The return value of the training function (i.e. res -- see code snippet below) is a tuple that has the following structure:
- first entry will be the path to the trained model on disk
- second entry will be the performance of the trained model using the training sequences
- third entry will be the performance of the trained model using the test sequences
During model training, we track the estimated average decoding error by plotting the generated avg_decodingerror_training file.
In [2]:
from pos_attr_extractor import *
seq = example()
attr_desc {'suffix_2': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'suffix of length 2 for the word observation'}, 'prefix_3': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'prefix of length 3 for the word observation'}, 'hascap': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'Orthographic feature -- hascap'}, 'suffix_3': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'suffix of length 3 for the word observation'}, 'hasdash': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'Orthographic feature -- hasdash'}, 'w': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'the word/token'}, 'prefix_2': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'prefix of length 2 for the word observation'}, 'hasnum': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'Orthographic feature -- hasnum'}, 'suffix_5': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'suffix of length 5 for the word observation'}, 'hasallcaps': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'Orthographic feature -- hasallcaps'}, 'prefix_5': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'prefix of length 5 for the word observation'}, 'suffix_4': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'suffix of length 4 for the word observation'}, 'prefix_4': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'prefix of length 4 for the word observation'}, 'shape': {'repr_func': <bound method GenericAttributeExtractor._represent_categorical_attr of <pos_attr_extractor.POS_AttributeExtractor object at 0x7fc4482d56a0>>, 'encoding': 'categorical', 'description': 'the shape of the word'}}
boundary (4, 4)
attributes {'suffix_2': 'nd', 'prefix_3': 'and', 'hascap': '0', 'suffix_3': 'and', 'hasdash': '0', 'w': 'and', 'prefix_2': 'an', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'aaa'}
--------------------------------------------------
boundary (13, 13)
attributes {'suffix_2': 'on', 'prefix_3': 'pro', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'production', 'prefix_2': 'pr', 'hasnum': '0', 'suffix_5': 'ction', 'hasallcaps': '0', 'prefix_5': 'produ', 'suffix_4': 'tion', 'prefix_4': 'prod', 'shape': 'aaaaaaaaaa'}
--------------------------------------------------
boundary (6, 6)
attributes {'suffix_2': '', 'prefix_3': '', 'hascap': '1', 'suffix_3': '', 'hasdash': '0', 'w': 'B', 'prefix_2': '', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '1', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'A'}
--------------------------------------------------
boundary (2, 2)
attributes {'suffix_2': 'ne', 'prefix_3': 'gen', 'hascap': '0', 'suffix_3': 'ene', 'hasdash': '0', 'w': 'gene', 'prefix_2': 'ge', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'gene', 'prefix_4': 'gene', 'shape': 'aaaa'}
--------------------------------------------------
boundary (12, 12)
attributes {'suffix_2': 'en', 'prefix_3': 'oxy', 'hascap': '0', 'suffix_3': 'gen', 'hasdash': '0', 'w': 'oxygen', 'prefix_2': 'ox', 'hasnum': '0', 'suffix_5': 'xygen', 'hasallcaps': '0', 'prefix_5': 'oxyge', 'suffix_4': 'ygen', 'prefix_4': 'oxyg', 'shape': 'aaaaaa'}
--------------------------------------------------
boundary (11, 11)
attributes {'suffix_2': 've', 'prefix_3': 'rea', 'hascap': '0', 'suffix_3': 'ive', 'hasdash': '0', 'w': 'reactive', 'prefix_2': 're', 'hasnum': '0', 'suffix_5': 'ctive', 'hasallcaps': '0', 'prefix_5': 'react', 'suffix_4': 'tive', 'prefix_4': 'reac', 'shape': 'aaaaaaaa'}
--------------------------------------------------
boundary (9, 9)
attributes {'suffix_2': '28', 'prefix_3': 'CD2', 'hascap': '1', 'suffix_3': 'D28', 'hasdash': '0', 'w': 'CD28', 'prefix_2': 'CD', 'hasnum': '1', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'CD28', 'prefix_4': 'CD28', 'shape': 'AADD'}
--------------------------------------------------
boundary (7, 7)
attributes {'suffix_2': 'on', 'prefix_3': 'act', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'activation', 'prefix_2': 'ac', 'hasnum': '0', 'suffix_5': 'ation', 'hasallcaps': '0', 'prefix_5': 'activ', 'suffix_4': 'tion', 'prefix_4': 'acti', 'shape': 'aaaaaaaaaa'}
--------------------------------------------------
boundary (1, 1)
attributes {'suffix_2': '-2', 'prefix_3': 'IL-', 'hascap': '1', 'suffix_3': 'L-2', 'hasdash': '1', 'w': 'IL-2', 'prefix_2': 'IL', 'hasnum': '1', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'IL-2', 'prefix_4': 'IL-2', 'shape': 'AA_D'}
--------------------------------------------------
boundary (8, 8)
attributes {'suffix_2': 'gh', 'prefix_3': 'thr', 'hascap': '0', 'suffix_3': 'ugh', 'hasdash': '0', 'w': 'through', 'prefix_2': 'th', 'hasnum': '0', 'suffix_5': 'rough', 'hasallcaps': '0', 'prefix_5': 'throu', 'suffix_4': 'ough', 'prefix_4': 'thro', 'shape': 'aaaaaaa'}
--------------------------------------------------
boundary (14, 14)
attributes {'suffix_2': 'by', 'prefix_3': '', 'hascap': '0', 'suffix_3': '', 'hasdash': '0', 'w': 'by', 'prefix_2': 'by', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'aa'}
--------------------------------------------------
boundary (3, 3)
attributes {'suffix_2': 'on', 'prefix_3': 'exp', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'expression', 'prefix_2': 'ex', 'hasnum': '0', 'suffix_5': 'ssion', 'hasallcaps': '0', 'prefix_5': 'expre', 'suffix_4': 'sion', 'prefix_4': 'expr', 'shape': 'aaaaaaaaaa'}
--------------------------------------------------
boundary (15, 15)
attributes {'suffix_2': 'se', 'prefix_3': '5-l', 'hascap': '0', 'suffix_3': 'ase', 'hasdash': '1', 'w': '5-lipoxygenase', 'prefix_2': '5-', 'hasnum': '1', 'suffix_5': 'enase', 'hasallcaps': '0', 'prefix_5': '5-lip', 'suffix_4': 'nase', 'prefix_4': '5-li', 'shape': 'D_aaaaaaaaaaaa'}
--------------------------------------------------
boundary (10, 10)
attributes {'suffix_2': 'es', 'prefix_3': 'req', 'hascap': '0', 'suffix_3': 'res', 'hasdash': '0', 'w': 'requires', 'prefix_2': 're', 'hasnum': '0', 'suffix_5': 'uires', 'hasallcaps': '0', 'prefix_5': 'requi', 'suffix_4': 'ires', 'prefix_4': 'requ', 'shape': 'aaaaaaaa'}
--------------------------------------------------
boundary (5, 5)
attributes {'suffix_2': 'pa', 'prefix_3': 'NF-', 'hascap': '1', 'suffix_3': 'ppa', 'hasdash': '1', 'w': 'NF-kappa', 'prefix_2': 'NF', 'hasnum': '0', 'suffix_5': 'kappa', 'hasallcaps': '0', 'prefix_5': 'NF-ka', 'suffix_4': 'appa', 'prefix_4': 'NF-k', 'shape': 'AA_aaaaa'}
--------------------------------------------------
seg_attr {(4, 4): {'suffix_2': 'nd', 'prefix_3': 'and', 'hascap': '0', 'suffix_3': 'and', 'hasdash': '0', 'w': 'and', 'prefix_2': 'an', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'aaa'}, (13, 13): {'suffix_2': 'on', 'prefix_3': 'pro', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'production', 'prefix_2': 'pr', 'hasnum': '0', 'suffix_5': 'ction', 'hasallcaps': '0', 'prefix_5': 'produ', 'suffix_4': 'tion', 'prefix_4': 'prod', 'shape': 'aaaaaaaaaa'}, (6, 6): {'suffix_2': '', 'prefix_3': '', 'hascap': '1', 'suffix_3': '', 'hasdash': '0', 'w': 'B', 'prefix_2': '', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '1', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'A'}, (2, 2): {'suffix_2': 'ne', 'prefix_3': 'gen', 'hascap': '0', 'suffix_3': 'ene', 'hasdash': '0', 'w': 'gene', 'prefix_2': 'ge', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'gene', 'prefix_4': 'gene', 'shape': 'aaaa'}, (12, 12): {'suffix_2': 'en', 'prefix_3': 'oxy', 'hascap': '0', 'suffix_3': 'gen', 'hasdash': '0', 'w': 'oxygen', 'prefix_2': 'ox', 'hasnum': '0', 'suffix_5': 'xygen', 'hasallcaps': '0', 'prefix_5': 'oxyge', 'suffix_4': 'ygen', 'prefix_4': 'oxyg', 'shape': 'aaaaaa'}, (11, 11): {'suffix_2': 've', 'prefix_3': 'rea', 'hascap': '0', 'suffix_3': 'ive', 'hasdash': '0', 'w': 'reactive', 'prefix_2': 're', 'hasnum': '0', 'suffix_5': 'ctive', 'hasallcaps': '0', 'prefix_5': 'react', 'suffix_4': 'tive', 'prefix_4': 'reac', 'shape': 'aaaaaaaa'}, (9, 9): {'suffix_2': '28', 'prefix_3': 'CD2', 'hascap': '1', 'suffix_3': 'D28', 'hasdash': '0', 'w': 'CD28', 'prefix_2': 'CD', 'hasnum': '1', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'CD28', 'prefix_4': 'CD28', 'shape': 'AADD'}, (7, 7): {'suffix_2': 'on', 'prefix_3': 'act', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'activation', 'prefix_2': 'ac', 'hasnum': '0', 'suffix_5': 'ation', 'hasallcaps': '0', 'prefix_5': 'activ', 'suffix_4': 'tion', 'prefix_4': 'acti', 'shape': 'aaaaaaaaaa'}, (1, 1): {'suffix_2': '-2', 'prefix_3': 'IL-', 'hascap': '1', 'suffix_3': 'L-2', 'hasdash': '1', 'w': 'IL-2', 'prefix_2': 'IL', 'hasnum': '1', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': 'IL-2', 'prefix_4': 'IL-2', 'shape': 'AA_D'}, (8, 8): {'suffix_2': 'gh', 'prefix_3': 'thr', 'hascap': '0', 'suffix_3': 'ugh', 'hasdash': '0', 'w': 'through', 'prefix_2': 'th', 'hasnum': '0', 'suffix_5': 'rough', 'hasallcaps': '0', 'prefix_5': 'throu', 'suffix_4': 'ough', 'prefix_4': 'thro', 'shape': 'aaaaaaa'}, (14, 14): {'suffix_2': 'by', 'prefix_3': '', 'hascap': '0', 'suffix_3': '', 'hasdash': '0', 'w': 'by', 'prefix_2': 'by', 'hasnum': '0', 'suffix_5': '', 'hasallcaps': '0', 'prefix_5': '', 'suffix_4': '', 'prefix_4': '', 'shape': 'aa'}, (3, 3): {'suffix_2': 'on', 'prefix_3': 'exp', 'hascap': '0', 'suffix_3': 'ion', 'hasdash': '0', 'w': 'expression', 'prefix_2': 'ex', 'hasnum': '0', 'suffix_5': 'ssion', 'hasallcaps': '0', 'prefix_5': 'expre', 'suffix_4': 'sion', 'prefix_4': 'expr', 'shape': 'aaaaaaaaaa'}, (15, 15): {'suffix_2': 'se', 'prefix_3': '5-l', 'hascap': '0', 'suffix_3': 'ase', 'hasdash': '1', 'w': '5-lipoxygenase', 'prefix_2': '5-', 'hasnum': '1', 'suffix_5': 'enase', 'hasallcaps': '0', 'prefix_5': '5-lip', 'suffix_4': 'nase', 'prefix_4': '5-li', 'shape': 'D_aaaaaaaaaaaa'}, (10, 10): {'suffix_2': 'es', 'prefix_3': 'req', 'hascap': '0', 'suffix_3': 'res', 'hasdash': '0', 'w': 'requires', 'prefix_2': 're', 'hasnum': '0', 'suffix_5': 'uires', 'hasallcaps': '0', 'prefix_5': 'requi', 'suffix_4': 'ires', 'prefix_4': 'requ', 'shape': 'aaaaaaaa'}, (5, 5): {'suffix_2': 'pa', 'prefix_3': 'NF-', 'hascap': '1', 'suffix_3': 'ppa', 'hasdash': '1', 'w': 'NF-kappa', 'prefix_2': 'NF', 'hasnum': '0', 'suffix_5': 'kappa', 'hasallcaps': '0', 'prefix_5': 'NF-ka', 'suffix_4': 'appa', 'prefix_4': 'NF-k', 'shape': 'AA_aaaaa'}}
In [3]:
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (10,6)
from pyseqlab.utilities import ReaderWriter
# import the module containing training workflow
# we use only 100 sequences for demonstration
# to go through the whole file simply omit passing the num_seqs keyword argument
num_seqs = 100
from train_tagger_workflow import *
optimization_options = {"method" : "COLLINS-PERCEPTRON",
"num_epochs":10,
'update_type':'max-fast',
'beam_size':5,
'shuffle_seq':True,
"tolerance":1e-6
}
# demonstrate training using only 100 sequences from the both the training and test file
res = run_training(optimization_options, template_config, num_seqs=num_seqs)
model_dir = res[0]
# using all sequences from the both the training and test file
# res = run_training(optimization_options, template_config)
# evaluating the performance of the models
# models_dir = res[0]
def plot_avg_decodingerror(model_dir):
# plot the estimated average decoding error during training phase
avg_decerror = ReaderWriter.read_data(os.path.join(model_dir, 'avg_decodingerror_training'))
plt.plot(avg_decerror[1:], label="method:{}, beam_size:{}, update_type:{}".format(optimization_options['method'],
optimization_options['beam_size'],
optimization_options['update_type']))
plt.legend(loc='upper right')
plt.xlabel('number of epochs')
plt.ylabel('estimated average decoding error')
plot_avg_decodingerror(model_dir)
1 sequences have been processed 2 sequences have been processed 3 sequences have been processed 4 sequences have been processed 5 sequences have been processed 6 sequences have been processed 7 sequences have been processed 8 sequences have been processed 9 sequences have been processed 10 sequences have been processed 11 sequences have been processed 12 sequences have been processed 13 sequences have been processed 14 sequences have been processed 15 sequences have been processed 16 sequences have been processed 17 sequences have been processed 18 sequences have been processed 19 sequences have been processed 20 sequences have been processed 21 sequences have been processed 22 sequences have been processed 23 sequences have been processed 24 sequences have been processed 25 sequences have been processed 26 sequences have been processed 27 sequences have been processed 28 sequences have been processed 29 sequences have been processed 30 sequences have been processed 31 sequences have been processed 32 sequences have been processed 33 sequences have been processed 34 sequences have been processed 35 sequences have been processed 36 sequences have been processed 37 sequences have been processed 38 sequences have been processed 39 sequences have been processed 40 sequences have been processed 41 sequences have been processed 42 sequences have been processed 43 sequences have been processed 44 sequences have been processed 45 sequences have been processed 46 sequences have been processed 47 sequences have been processed 48 sequences have been processed 49 sequences have been processed 50 sequences have been processed 51 sequences have been processed 52 sequences have been processed 53 sequences have been processed 54 sequences have been processed 55 sequences have been processed 56 sequences have been processed 57 sequences have been processed 58 sequences have been processed 59 sequences have been processed 60 sequences have been processed 61 sequences have been processed 62 sequences have been processed 63 sequences have been processed 64 sequences have been processed 65 sequences have been processed 66 sequences have been processed 67 sequences have been processed 68 sequences have been processed 69 sequences have been processed 70 sequences have been processed 71 sequences have been processed 72 sequences have been processed 73 sequences have been processed 74 sequences have been processed 75 sequences have been processed 76 sequences have been processed 77 sequences have been processed 78 sequences have been processed 79 sequences have been processed 80 sequences have been processed 81 sequences have been processed 82 sequences have been processed 83 sequences have been processed 84 sequences have been processed 85 sequences have been processed 86 sequences have been processed 87 sequences have been processed 88 sequences have been processed 89 sequences have been processed 90 sequences have been processed 91 sequences have been processed 92 sequences have been processed 93 sequences have been processed 94 sequences have been processed 95 sequences have been processed 96 sequences have been processed 97 sequences have been processed 98 sequences have been processed 99 sequences have been processed 100 sequences have been processed dumping globalfeatures -- processed seqs: 1 dumping globalfeatures -- processed seqs: 2 dumping globalfeatures -- processed seqs: 3 dumping globalfeatures -- processed seqs: 4 dumping globalfeatures -- processed seqs: 5 dumping globalfeatures -- processed seqs: 6 dumping globalfeatures -- processed seqs: 7 dumping globalfeatures -- processed seqs: 8 dumping globalfeatures -- processed seqs: 9 dumping globalfeatures -- processed seqs: 10 dumping globalfeatures -- processed seqs: 11 dumping globalfeatures -- processed seqs: 12 dumping globalfeatures -- processed seqs: 13 dumping globalfeatures -- processed seqs: 14 dumping globalfeatures -- processed seqs: 15 dumping globalfeatures -- processed seqs: 16 dumping globalfeatures -- processed seqs: 17 dumping globalfeatures -- processed seqs: 18 dumping globalfeatures -- processed seqs: 19 dumping globalfeatures -- processed seqs: 20 dumping globalfeatures -- processed seqs: 21 dumping globalfeatures -- processed seqs: 22 dumping globalfeatures -- processed seqs: 23 dumping globalfeatures -- processed seqs: 24 dumping globalfeatures -- processed seqs: 25 dumping globalfeatures -- processed seqs: 26 dumping globalfeatures -- processed seqs: 27 dumping globalfeatures -- processed seqs: 28 dumping globalfeatures -- processed seqs: 29 dumping globalfeatures -- processed seqs: 30 dumping globalfeatures -- processed seqs: 31 dumping globalfeatures -- processed seqs: 32 dumping globalfeatures -- processed seqs: 33 dumping globalfeatures -- processed seqs: 34 dumping globalfeatures -- processed seqs: 35 dumping globalfeatures -- processed seqs: 36 dumping globalfeatures -- processed seqs: 37 dumping globalfeatures -- processed seqs: 38 dumping globalfeatures -- processed seqs: 39 dumping globalfeatures -- processed seqs: 40 dumping globalfeatures -- processed seqs: 41 dumping globalfeatures -- processed seqs: 42 dumping globalfeatures -- processed seqs: 43 dumping globalfeatures -- processed seqs: 44 dumping globalfeatures -- processed seqs: 45 dumping globalfeatures -- processed seqs: 46 dumping globalfeatures -- processed seqs: 47 dumping globalfeatures -- processed seqs: 48 dumping globalfeatures -- processed seqs: 49 dumping globalfeatures -- processed seqs: 50 dumping globalfeatures -- processed seqs: 51 dumping globalfeatures -- processed seqs: 52 dumping globalfeatures -- processed seqs: 53 dumping globalfeatures -- processed seqs: 54 dumping globalfeatures -- processed seqs: 55 dumping globalfeatures -- processed seqs: 56 dumping globalfeatures -- processed seqs: 57 dumping globalfeatures -- processed seqs: 58 dumping globalfeatures -- processed seqs: 59 dumping globalfeatures -- processed seqs: 60 dumping globalfeatures -- processed seqs: 61 dumping globalfeatures -- processed seqs: 62 dumping globalfeatures -- processed seqs: 63 dumping globalfeatures -- processed seqs: 64 dumping globalfeatures -- processed seqs: 65 dumping globalfeatures -- processed seqs: 66 dumping globalfeatures -- processed seqs: 67 dumping globalfeatures -- processed seqs: 68 dumping globalfeatures -- processed seqs: 69 dumping globalfeatures -- processed seqs: 70 dumping globalfeatures -- processed seqs: 71 dumping globalfeatures -- processed seqs: 72 dumping globalfeatures -- processed seqs: 73 dumping globalfeatures -- processed seqs: 74 dumping globalfeatures -- processed seqs: 75 dumping globalfeatures -- processed seqs: 76 dumping globalfeatures -- processed seqs: 77 dumping globalfeatures -- processed seqs: 78 dumping globalfeatures -- processed seqs: 79 dumping globalfeatures -- processed seqs: 80 dumping globalfeatures -- processed seqs: 81 dumping globalfeatures -- processed seqs: 82 dumping globalfeatures -- processed seqs: 83 dumping globalfeatures -- processed seqs: 84 dumping globalfeatures -- processed seqs: 85 dumping globalfeatures -- processed seqs: 86 dumping globalfeatures -- processed seqs: 87 dumping globalfeatures -- processed seqs: 88 dumping globalfeatures -- processed seqs: 89 dumping globalfeatures -- processed seqs: 90 dumping globalfeatures -- processed seqs: 91 dumping globalfeatures -- processed seqs: 92 dumping globalfeatures -- processed seqs: 93 dumping globalfeatures -- processed seqs: 94 dumping globalfeatures -- processed seqs: 95 dumping globalfeatures -- processed seqs: 96 dumping globalfeatures -- processed seqs: 97 dumping globalfeatures -- processed seqs: 98 dumping globalfeatures -- processed seqs: 99 dumping globalfeatures -- processed seqs: 100 constructing model -- processed seqs: 1 constructing model -- processed seqs: 2 constructing model -- processed seqs: 3 constructing model -- processed seqs: 4 constructing model -- processed seqs: 5 constructing model -- processed seqs: 6 constructing model -- processed seqs: 7 constructing model -- processed seqs: 8 constructing model -- processed seqs: 9 constructing model -- processed seqs: 10 constructing model -- processed seqs: 11 constructing model -- processed seqs: 12 constructing model -- processed seqs: 13 constructing model -- processed seqs: 14 constructing model -- processed seqs: 15 constructing model -- processed seqs: 16 constructing model -- processed seqs: 17 constructing model -- processed seqs: 18 constructing model -- processed seqs: 19 constructing model -- processed seqs: 20 constructing model -- processed seqs: 21 constructing model -- processed seqs: 22 constructing model -- processed seqs: 23 constructing model -- processed seqs: 24 constructing model -- processed seqs: 25 constructing model -- processed seqs: 26 constructing model -- processed seqs: 27 constructing model -- processed seqs: 28 constructing model -- processed seqs: 29 constructing model -- processed seqs: 30 constructing model -- processed seqs: 31 constructing model -- processed seqs: 32 constructing model -- processed seqs: 33 constructing model -- processed seqs: 34 constructing model -- processed seqs: 35 constructing model -- processed seqs: 36 constructing model -- processed seqs: 37 constructing model -- processed seqs: 38 constructing model -- processed seqs: 39 constructing model -- processed seqs: 40 constructing model -- processed seqs: 41 constructing model -- processed seqs: 42 constructing model -- processed seqs: 43 constructing model -- processed seqs: 44 constructing model -- processed seqs: 45 constructing model -- processed seqs: 46 constructing model -- processed seqs: 47 constructing model -- processed seqs: 48 constructing model -- processed seqs: 49 constructing model -- processed seqs: 50 constructing model -- processed seqs: 51 constructing model -- processed seqs: 52 constructing model -- processed seqs: 53 constructing model -- processed seqs: 54 constructing model -- processed seqs: 55 constructing model -- processed seqs: 56 constructing model -- processed seqs: 57 constructing model -- processed seqs: 58 constructing model -- processed seqs: 59 constructing model -- processed seqs: 60 constructing model -- processed seqs: 61 constructing model -- processed seqs: 62 constructing model -- processed seqs: 63 constructing model -- processed seqs: 64 constructing model -- processed seqs: 65 constructing model -- processed seqs: 66 constructing model -- processed seqs: 67 constructing model -- processed seqs: 68 constructing model -- processed seqs: 69 constructing model -- processed seqs: 70 constructing model -- processed seqs: 71 constructing model -- processed seqs: 72 constructing model -- processed seqs: 73 constructing model -- processed seqs: 74 constructing model -- processed seqs: 75 constructing model -- processed seqs: 76 constructing model -- processed seqs: 77 constructing model -- processed seqs: 78 constructing model -- processed seqs: 79 constructing model -- processed seqs: 80 constructing model -- processed seqs: 81 constructing model -- processed seqs: 82 constructing model -- processed seqs: 83 constructing model -- processed seqs: 84 constructing model -- processed seqs: 85 constructing model -- processed seqs: 86 constructing model -- processed seqs: 87 constructing model -- processed seqs: 88 constructing model -- processed seqs: 89 constructing model -- processed seqs: 90 identifying model active features -- processed seqs: 1 identifying model active features -- processed seqs: 2 identifying model active features -- processed seqs: 3 identifying model active features -- processed seqs: 4 identifying model active features -- processed seqs: 5 identifying model active features -- processed seqs: 6 identifying model active features -- processed seqs: 7 identifying model active features -- processed seqs: 8 identifying model active features -- processed seqs: 9 identifying model active features -- processed seqs: 10 identifying model active features -- processed seqs: 11 identifying model active features -- processed seqs: 12 identifying model active features -- processed seqs: 13 identifying model active features -- processed seqs: 14 identifying model active features -- processed seqs: 15 identifying model active features -- processed seqs: 16 identifying model active features -- processed seqs: 17 identifying model active features -- processed seqs: 18 identifying model active features -- processed seqs: 19 identifying model active features -- processed seqs: 20 identifying model active features -- processed seqs: 21 identifying model active features -- processed seqs: 22 identifying model active features -- processed seqs: 23 identifying model active features -- processed seqs: 24 identifying model active features -- processed seqs: 25 identifying model active features -- processed seqs: 26 identifying model active features -- processed seqs: 27 identifying model active features -- processed seqs: 28 identifying model active features -- processed seqs: 29 identifying model active features -- processed seqs: 30 identifying model active features -- processed seqs: 31 identifying model active features -- processed seqs: 32 identifying model active features -- processed seqs: 33 identifying model active features -- processed seqs: 34 identifying model active features -- processed seqs: 35 identifying model active features -- processed seqs: 36 identifying model active features -- processed seqs: 37 identifying model active features -- processed seqs: 38 identifying model active features -- processed seqs: 39 identifying model active features -- processed seqs: 40 identifying model active features -- processed seqs: 41 identifying model active features -- processed seqs: 42 identifying model active features -- processed seqs: 43 identifying model active features -- processed seqs: 44 identifying model active features -- processed seqs: 45 identifying model active features -- processed seqs: 46 identifying model active features -- processed seqs: 47 identifying model active features -- processed seqs: 48 identifying model active features -- processed seqs: 49 identifying model active features -- processed seqs: 50 identifying model active features -- processed seqs: 51 identifying model active features -- processed seqs: 52 identifying model active features -- processed seqs: 53 identifying model active features -- processed seqs: 54 identifying model active features -- processed seqs: 55 identifying model active features -- processed seqs: 56 identifying model active features -- processed seqs: 57 identifying model active features -- processed seqs: 58 identifying model active features -- processed seqs: 59 identifying model active features -- processed seqs: 60 identifying model active features -- processed seqs: 61 identifying model active features -- processed seqs: 62 identifying model active features -- processed seqs: 63 identifying model active features -- processed seqs: 64 identifying model active features -- processed seqs: 65 identifying model active features -- processed seqs: 66 identifying model active features -- processed seqs: 67 identifying model active features -- processed seqs: 68 identifying model active features -- processed seqs: 69 identifying model active features -- processed seqs: 70 identifying model active features -- processed seqs: 71 identifying model active features -- processed seqs: 72 identifying model active features -- processed seqs: 73 identifying model active features -- processed seqs: 74 identifying model active features -- processed seqs: 75 identifying model active features -- processed seqs: 76 identifying model active features -- processed seqs: 77 identifying model active features -- processed seqs: 78 identifying model active features -- processed seqs: 79 identifying model active features -- processed seqs: 80 identifying model active features -- processed seqs: 81 identifying model active features -- processed seqs: 82 identifying model active features -- processed seqs: 83 identifying model active features -- processed seqs: 84 identifying model active features -- processed seqs: 85 identifying model active features -- processed seqs: 86 identifying model active features -- processed seqs: 87 identifying model active features -- processed seqs: 88 identifying model active features -- processed seqs: 89 identifying model active features -- processed seqs: 90 identifying model active features -- processed seqs: 91 identifying model active features -- processed seqs: 92 identifying model active features -- processed seqs: 93 identifying model active features -- processed seqs: 94 identifying model active features -- processed seqs: 95 identifying model active features -- processed seqs: 96 identifying model active features -- processed seqs: 97 identifying model active features -- processed seqs: 98 identifying model active features -- processed seqs: 99 identifying model active features -- processed seqs: 100 sequences left 90 in max-fast update routine ... sequences left 89 in max-fast update routine ... sequences left 88 in max-fast update routine ... sequences left 87 in max-fast update routine ... sequences left 86 in max-fast update routine ... sequences left 85 in max-fast update routine ... sequences left 84 in max-fast update routine ... sequences left 83 in max-fast update routine ... sequences left 82 in max-fast update routine ... sequences left 81 in max-fast update routine ... sequences left 80 in max-fast update routine ... sequences left 79 in max-fast update routine ... sequences left 78 in max-fast update routine ... sequences left 77 in max-fast update routine ... sequences left 76 in max-fast update routine ... sequences left 75 in max-fast update routine ... sequences left 74 in max-fast update routine ... sequences left 73 in max-fast update routine ... sequences left 72 in max-fast update routine ... sequences left 71 in max-fast update routine ... sequences left 70 in max-fast update routine ... sequences left 69 in max-fast update routine ... sequences left 68 in max-fast update routine ... sequences left 67 in max-fast update routine ... sequences left 66 in max-fast update routine ... sequences left 65 in max-fast update routine ... sequences left 64 in max-fast update routine ... sequences left 63 in max-fast update routine ... sequences left 62 in max-fast update routine ... sequences left 61 in max-fast update routine ... sequences left 60 in max-fast update routine ... sequences left 59 in full update routine ... sequences left 58 in max-fast update routine ... sequences left 57 in max-fast update routine ... sequences left 56 in max-fast update routine ... sequences left 55 in max-fast update routine ... sequences left 54 in max-fast update routine ... sequences left 53 in max-fast update routine ... sequences left 52 in max-fast update routine ... sequences left 51 in max-fast update routine ... sequences left 50 in max-fast update routine ... sequences left 49 in max-fast update routine ... sequences left 48 in max-fast update routine ... sequences left 47 in max-fast update routine ... sequences left 46 in max-fast update routine ... sequences left 45 in max-fast update routine ... sequences left 44 in max-fast update routine ... sequences left 43 in max-fast update routine ... sequences left 42 in max-fast update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in max-fast update routine ... sequences left 36 in max-fast update routine ... sequences left 35 in max-fast update routine ... sequences left 34 in max-fast update routine ... sequences left 33 in max-fast update routine ... sequences left 32 in full update routine ... sequences left 31 in max-fast update routine ... sequences left 30 in max-fast update routine ... sequences left 29 in max-fast update routine ... sequences left 28 in max-fast update routine ... sequences left 27 in max-fast update routine ... sequences left 26 in max-fast update routine ... sequences left 25 in max-fast update routine ... sequences left 24 in max-fast update routine ... sequences left 23 in max-fast update routine ... sequences left 22 in max-fast update routine ... sequences left 21 in max-fast update routine ... sequences left 20 in max-fast update routine ... sequences left 19 in max-fast update routine ... sequences left 18 in full update routine ... sequences left 17 in max-fast update routine ... sequences left 16 in max-fast update routine ... sequences left 15 in max-fast update routine ... sequences left 14 in max-fast update routine ... sequences left 13 in max-fast update routine ... sequences left 12 in max-fast update routine ... sequences left 11 in full update routine ... sequences left 10 in max-fast update routine ... sequences left 9 in max-fast update routine ... sequences left 8 in full update routine ... sequences left 7 in max-fast update routine ... sequences left 6 in max-fast update routine ... sequences left 5 in max-fast update routine ... sequences left 4 in full update routine ... sequences left 3 in max-fast update routine ... sequences left 2 in max-fast update routine ... sequences left 1 in max-fast update routine ... reldiff = 1.0 average error : [0, 0.39336301929187834] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in max-fast update routine ... sequences left 88 in max-fast update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in max-fast update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in max-fast update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in max-fast update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in max-fast update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in max-fast update routine ... sequences left 56 in max-fast update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in max-fast update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in max-fast update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in max-fast update routine ... sequences left 18 in max-fast update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in max-fast update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in max-fast update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.7825977278672305 average error : [0, 0.39336301929187834, 0.04797381530906465] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in max-fast update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in max-fast update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in max-fast update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in max-fast update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in max-fast update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in max-fast update routine ... sequences left 1 in full update routine ... reldiff = 0.4859973050225044 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in max-fast update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in max-fast update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in max-fast update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.11563648683673235 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.8819256756814069 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.6893189374976649 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577, 0.00024154589371980676] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.24324324324324326 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577, 0.00024154589371980676, 0.0003968253968253968] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.5022920030436316 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577, 0.00024154589371980676, 0.0003968253968253968, 0.0011977858983601396] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... reldiff = 0.5809785708147466 average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577, 0.00024154589371980676, 0.0003968253968253968, 0.0011977858983601396, 0.00031746031746031746] self._exitloop False sequences left 90 in full update routine ... sequences left 89 in full update routine ... sequences left 88 in full update routine ... sequences left 87 in full update routine ... sequences left 86 in full update routine ... sequences left 85 in full update routine ... sequences left 84 in full update routine ... sequences left 83 in full update routine ... sequences left 82 in full update routine ... sequences left 81 in full update routine ... sequences left 80 in full update routine ... sequences left 79 in full update routine ... sequences left 78 in full update routine ... sequences left 77 in full update routine ... sequences left 76 in full update routine ... sequences left 75 in full update routine ... sequences left 74 in full update routine ... sequences left 73 in full update routine ... sequences left 72 in full update routine ... sequences left 71 in full update routine ... sequences left 70 in full update routine ... sequences left 69 in full update routine ... sequences left 68 in full update routine ... sequences left 67 in full update routine ... sequences left 66 in full update routine ... sequences left 65 in full update routine ... sequences left 64 in full update routine ... sequences left 63 in full update routine ... sequences left 62 in full update routine ... sequences left 61 in full update routine ... sequences left 60 in full update routine ... sequences left 59 in full update routine ... sequences left 58 in full update routine ... sequences left 57 in full update routine ... sequences left 56 in full update routine ... sequences left 55 in full update routine ... sequences left 54 in full update routine ... sequences left 53 in full update routine ... sequences left 52 in full update routine ... sequences left 51 in full update routine ... sequences left 50 in full update routine ... sequences left 49 in full update routine ... sequences left 48 in full update routine ... sequences left 47 in full update routine ... sequences left 46 in full update routine ... sequences left 45 in full update routine ... sequences left 44 in full update routine ... sequences left 43 in full update routine ... sequences left 42 in full update routine ... sequences left 41 in full update routine ... sequences left 40 in full update routine ... sequences left 39 in full update routine ... sequences left 38 in full update routine ... sequences left 37 in full update routine ... sequences left 36 in full update routine ... sequences left 35 in full update routine ... sequences left 34 in full update routine ... sequences left 33 in full update routine ... sequences left 32 in full update routine ... sequences left 31 in full update routine ... sequences left 30 in full update routine ... sequences left 29 in full update routine ... sequences left 28 in full update routine ... sequences left 27 in full update routine ... sequences left 26 in full update routine ... sequences left 25 in full update routine ... sequences left 24 in full update routine ... sequences left 23 in full update routine ... sequences left 22 in full update routine ... sequences left 21 in full update routine ... sequences left 20 in full update routine ... sequences left 19 in full update routine ... sequences left 18 in full update routine ... sequences left 17 in full update routine ... sequences left 16 in full update routine ... sequences left 15 in full update routine ... sequences left 14 in full update routine ... sequences left 13 in full update routine ... sequences left 12 in full update routine ... sequences left 11 in full update routine ... sequences left 10 in full update routine ... sequences left 9 in full update routine ... sequences left 8 in full update routine ... sequences left 7 in full update routine ... sequences left 6 in full update routine ... sequences left 5 in full update routine ... sequences left 4 in full update routine ... sequences left 3 in full update routine ... sequences left 2 in full update routine ... sequences left 1 in full update routine ... average error : [0, 0.39336301929187834, 0.04797381530906465, 0.016594020913677517, 0.020933580952940865, 0.0013133985356207577, 0.00024154589371980676, 0.0003968253968253968, 0.0011977858983601396, 0.00031746031746031746, 0.0] self._exitloop True sequence decoded -- 89 sequences are left sequence decoded -- 88 sequences are left sequence decoded -- 87 sequences are left sequence decoded -- 86 sequences are left sequence decoded -- 85 sequences are left sequence decoded -- 84 sequences are left sequence decoded -- 83 sequences are left sequence decoded -- 82 sequences are left sequence decoded -- 81 sequences are left sequence decoded -- 80 sequences are left sequence decoded -- 79 sequences are left sequence decoded -- 78 sequences are left sequence decoded -- 77 sequences are left sequence decoded -- 76 sequences are left sequence decoded -- 75 sequences are left sequence decoded -- 74 sequences are left sequence decoded -- 73 sequences are left sequence decoded -- 72 sequences are left sequence decoded -- 71 sequences are left sequence decoded -- 70 sequences are left sequence decoded -- 69 sequences are left sequence decoded -- 68 sequences are left sequence decoded -- 67 sequences are left sequence decoded -- 66 sequences are left sequence decoded -- 65 sequences are left sequence decoded -- 64 sequences are left sequence decoded -- 63 sequences are left sequence decoded -- 62 sequences are left sequence decoded -- 61 sequences are left sequence decoded -- 60 sequences are left sequence decoded -- 59 sequences are left sequence decoded -- 58 sequences are left sequence decoded -- 57 sequences are left sequence decoded -- 56 sequences are left sequence decoded -- 55 sequences are left sequence decoded -- 54 sequences are left sequence decoded -- 53 sequences are left sequence decoded -- 52 sequences are left sequence decoded -- 51 sequences are left sequence decoded -- 50 sequences are left sequence decoded -- 49 sequences are left sequence decoded -- 48 sequences are left sequence decoded -- 47 sequences are left sequence decoded -- 46 sequences are left sequence decoded -- 45 sequences are left sequence decoded -- 44 sequences are left sequence decoded -- 43 sequences are left sequence decoded -- 42 sequences are left sequence decoded -- 41 sequences are left sequence decoded -- 40 sequences are left sequence decoded -- 39 sequences are left sequence decoded -- 38 sequences are left sequence decoded -- 37 sequences are left sequence decoded -- 36 sequences are left sequence decoded -- 35 sequences are left sequence decoded -- 34 sequences are left sequence decoded -- 33 sequences are left sequence decoded -- 32 sequences are left sequence decoded -- 31 sequences are left sequence decoded -- 30 sequences are left sequence decoded -- 29 sequences are left sequence decoded -- 28 sequences are left sequence decoded -- 27 sequences are left sequence decoded -- 26 sequences are left sequence decoded -- 25 sequences are left sequence decoded -- 24 sequences are left sequence decoded -- 23 sequences are left sequence decoded -- 22 sequences are left sequence decoded -- 21 sequences are left sequence decoded -- 20 sequences are left sequence decoded -- 19 sequences are left sequence decoded -- 18 sequences are left sequence decoded -- 17 sequences are left sequence decoded -- 16 sequences are left sequence decoded -- 15 sequences are left sequence decoded -- 14 sequences are left sequence decoded -- 13 sequences are left sequence decoded -- 12 sequences are left sequence decoded -- 11 sequences are left sequence decoded -- 10 sequences are left sequence decoded -- 9 sequences are left sequence decoded -- 8 sequences are left sequence decoded -- 7 sequences are left sequence decoded -- 6 sequences are left sequence decoded -- 5 sequences are left sequence decoded -- 4 sequences are left sequence decoded -- 3 sequences are left sequence decoded -- 2 sequences are left sequence decoded -- 1 sequences are left sequence decoded -- 0 sequences are left f1 0.9972972972972973 F1 score on the training set: 0.997297297297 sequence decoded -- 9 sequences are left sequence decoded -- 8 sequences are left sequence decoded -- 7 sequences are left sequence decoded -- 6 sequences are left sequence decoded -- 5 sequences are left sequence decoded -- 4 sequences are left sequence decoded -- 3 sequences are left sequence decoded -- 2 sequences are left sequence decoded -- 1 sequences are left sequence decoded -- 0 sequences are left f1 0.909871244635193 F1 score on the test set: 0.909871244635
3. Trained model evaluation¶
As part of the training workflow we defined earlier (see here), once the model training ended, we used the trained model to decode the training (90% share) and test (10% share) sequences in addition to reporting performance evaluation. The performance evaluation is demonstrated again in (this code snippet). To verify the performance evaluation computation, we will use the conlleval.pl script to compare both results (see this code section).
As a reminder, a trained model folder will have the following structure:
├── avg_decodingerror_training ├── crf_training_log.txt ├── decoding_seqs │ ├── test_fold_0.txt │ ├── train_fold_0.txt ├── model_parts │ ├── class_desc.txt │ ├── FE_templateX │ ├── FE_templateY │ ├── MR_L │ ├── MR_modelfeatures │ ├── MR_modelfeaturescodebook │ ├── MR_Ycodebook │ ├── weights
Each model has a model_parts folder. The decoded sequences (i.e. decoded training and test files) are found under decoding_seqs folder.
In [4]:
# model_dir is the path to the trained model (i.e. the model we just trained earlier)
crf_percep = revive_learnedmodel(model_dir)
data_type = ('training', 'test')
for i in range(1,3):
taglevel_perf = res[i]['taglevel_confusion_matrix']
print("evaluating the performance on {} data".format(data_type[i-1]))
print("f1:", res[i]['f1'])
for state, code in crf_percep.model.Y_codebook.items():
print("confusion_matrix for state={}: ".format(state))
print(taglevel_perf[code])
print("-"*40)
print()
evaluating the performance on training data f1: 0.997297297297 confusion_matrix for state=VB: [[ 29. 0.] [ 0. 2191.]] ---------------------------------------- confusion_matrix for state=NNS: [[ 1.48000000e+02 0.00000000e+00] [ 1.00000000e+00 2.07100000e+03]] ---------------------------------------- confusion_matrix for state=,: [[ 76. 0.] [ 0. 2144.]] ---------------------------------------- confusion_matrix for state=.: [[ 90. 0.] [ 0. 2130.]] ---------------------------------------- confusion_matrix for state=FW: [[ 10. 0.] [ 0. 2210.]] ---------------------------------------- confusion_matrix for state=:: [[ 3. 0.] [ 0. 2217.]] ---------------------------------------- confusion_matrix for state=CD: [[ 12. 0.] [ 0. 2208.]] ---------------------------------------- confusion_matrix for state=IN: [[ 289. 2.] [ 0. 1929.]] ---------------------------------------- confusion_matrix for state=VBD: [[ 19. 0.] [ 0. 2201.]] ---------------------------------------- confusion_matrix for state=MD: [[ 9. 0.] [ 0. 2211.]] ---------------------------------------- confusion_matrix for state=WRB: [[ 2.00000000e+00 0.00000000e+00] [ 0.00000000e+00 2.21800000e+03]] ---------------------------------------- confusion_matrix for state=NNP: [[ 2.00000000e+00 0.00000000e+00] [ 0.00000000e+00 2.21800000e+03]] ---------------------------------------- confusion_matrix for state=DT: [[ 1.75000000e+02 0.00000000e+00] [ 1.00000000e+00 2.04400000e+03]] ---------------------------------------- confusion_matrix for state=JJS: [[ 1.00000000e+00 0.00000000e+00] [ 0.00000000e+00 2.21900000e+03]] ---------------------------------------- confusion_matrix for state=VBP: [[ 5.00000000e+01 0.00000000e+00] [ 1.00000000e+00 2.16900000e+03]] ---------------------------------------- confusion_matrix for state=PRP: [[ 21. 0.] [ 0. 2199.]] ---------------------------------------- confusion_matrix for state=JJ: [[ 2.04000000e+02 1.00000000e+00] [ 1.00000000e+00 2.01400000e+03]] ---------------------------------------- confusion_matrix for state=): [[ 17. 0.] [ 0. 2203.]] ---------------------------------------- confusion_matrix for state=VBN: [[ 74. 0.] [ 0. 2146.]] ---------------------------------------- confusion_matrix for state=TO: [[ 35. 0.] [ 0. 2185.]] ---------------------------------------- confusion_matrix for state=RB: [[ 5.90000000e+01 1.00000000e+00] [ 0.00000000e+00 2.16000000e+03]] ---------------------------------------- confusion_matrix for state=RBS: [[ 1.00000000e+00 0.00000000e+00] [ 0.00000000e+00 2.21900000e+03]] ---------------------------------------- confusion_matrix for state=(: [[ 17. 0.] [ 0. 2203.]] ---------------------------------------- confusion_matrix for state=PRP$: [[ 6. 0.] [ 0. 2214.]] ---------------------------------------- confusion_matrix for state=NN: [[ 6.78000000e+02 2.00000000e+00] [ 1.00000000e+00 1.53900000e+03]] ---------------------------------------- confusion_matrix for state=VBZ: [[ 6.70000000e+01 0.00000000e+00] [ 1.00000000e+00 2.15200000e+03]] ---------------------------------------- confusion_matrix for state=EX: [[ 2.00000000e+00 0.00000000e+00] [ 0.00000000e+00 2.21800000e+03]] ---------------------------------------- confusion_matrix for state=CC: [[ 78. 0.] [ 0. 2142.]] ---------------------------------------- confusion_matrix for state=VBG: [[ 21. 0.] [ 0. 2199.]] ---------------------------------------- confusion_matrix for state=WDT: [[ 19. 0.] [ 0. 2201.]] ---------------------------------------- evaluating the performance on test data f1: 0.909871244635 confusion_matrix for state=VB: [[ 1. 1.] [ 0. 231.]] ---------------------------------------- confusion_matrix for state=NNS: [[ 13. 0.] [ 5. 215.]] ---------------------------------------- confusion_matrix for state=,: [[ 7. 0.] [ 0. 226.]] ---------------------------------------- confusion_matrix for state=.: [[ 10. 0.] [ 0. 223.]] ---------------------------------------- confusion_matrix for state=FW: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=:: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=CD: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=IN: [[ 29. 0.] [ 0. 204.]] ---------------------------------------- confusion_matrix for state=VBD: [[ 3. 0.] [ 1. 229.]] ---------------------------------------- confusion_matrix for state=MD: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=WRB: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=NNP: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=DT: [[ 16. 0.] [ 0. 217.]] ---------------------------------------- confusion_matrix for state=JJS: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=VBP: [[ 3. 0.] [ 1. 229.]] ---------------------------------------- confusion_matrix for state=PRP: [[ 2. 0.] [ 0. 231.]] ---------------------------------------- confusion_matrix for state=JJ: [[ 18. 2.] [ 8. 205.]] ---------------------------------------- confusion_matrix for state=): [[ 4. 1.] [ 0. 228.]] ---------------------------------------- confusion_matrix for state=VBN: [[ 8. 4.] [ 0. 221.]] ---------------------------------------- confusion_matrix for state=TO: [[ 6. 0.] [ 0. 227.]] ---------------------------------------- confusion_matrix for state=RB: [[ 3. 0.] [ 1. 229.]] ---------------------------------------- confusion_matrix for state=RBS: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=(: [[ 4. 0.] [ 0. 229.]] ---------------------------------------- confusion_matrix for state=PRP$: [[ 0. 0.] [ 1. 232.]] ---------------------------------------- confusion_matrix for state=NN: [[ 66. 13.] [ 2. 152.]] ---------------------------------------- confusion_matrix for state=VBZ: [[ 3. 0.] [ 1. 229.]] ---------------------------------------- confusion_matrix for state=EX: [[ 0. 0.] [ 0. 233.]] ---------------------------------------- confusion_matrix for state=CC: [[ 9. 0.] [ 0. 224.]] ---------------------------------------- confusion_matrix for state=VBG: [[ 5. 0.] [ 0. 228.]] ---------------------------------------- confusion_matrix for state=WDT: [[ 2. 0.] [ 0. 231.]] ----------------------------------------
In [5]:
# using conlleval.pl script to verify our performance computation
for f in ('train_fold_0.txt', 'test_fold_0.txt'):
dec_file = os.path.join(model_dir, 'decoding_seqs', f)
# -r option specifies takes raw input -- no BIO/IOB representation
# -d option specifies the separator (in our case the file is tab delimited)
! ./conlleval.pl -r -d '\t' < $dec_file
print()
processed 2220 tokens with 2220 phrases; found: 2220 phrases; correct: 2214.
accuracy: 99.73%; precision: 99.73%; recall: 99.73%; FB1: 99.73
(: precision: 100.00%; recall: 100.00%; FB1: 100.00 17
): precision: 100.00%; recall: 100.00%; FB1: 100.00 17
,: precision: 100.00%; recall: 100.00%; FB1: 100.00 76
.: precision: 100.00%; recall: 100.00%; FB1: 100.00 90
:: precision: 100.00%; recall: 100.00%; FB1: 100.00 3
CC: precision: 100.00%; recall: 100.00%; FB1: 100.00 78
CD: precision: 100.00%; recall: 100.00%; FB1: 100.00 12
DT: precision: 100.00%; recall: 99.43%; FB1: 99.72 175
EX: precision: 100.00%; recall: 100.00%; FB1: 100.00 2
FW: precision: 100.00%; recall: 100.00%; FB1: 100.00 10
IN: precision: 99.31%; recall: 100.00%; FB1: 99.66 291
JJ: precision: 99.51%; recall: 99.51%; FB1: 99.51 205
JJS: precision: 100.00%; recall: 100.00%; FB1: 100.00 1
MD: precision: 100.00%; recall: 100.00%; FB1: 100.00 9
NN: precision: 99.71%; recall: 99.85%; FB1: 99.78 680
NNP: precision: 100.00%; recall: 100.00%; FB1: 100.00 2
NNS: precision: 100.00%; recall: 99.33%; FB1: 99.66 148
PRP: precision: 100.00%; recall: 100.00%; FB1: 100.00 21
PRP$: precision: 100.00%; recall: 100.00%; FB1: 100.00 6
RB: precision: 98.33%; recall: 100.00%; FB1: 99.16 60
RBS: precision: 100.00%; recall: 100.00%; FB1: 100.00 1
TO: precision: 100.00%; recall: 100.00%; FB1: 100.00 35
VB: precision: 100.00%; recall: 100.00%; FB1: 100.00 29
VBD: precision: 100.00%; recall: 100.00%; FB1: 100.00 19
VBG: precision: 100.00%; recall: 100.00%; FB1: 100.00 21
VBN: precision: 100.00%; recall: 100.00%; FB1: 100.00 74
VBP: precision: 100.00%; recall: 98.04%; FB1: 99.01 50
VBZ: precision: 100.00%; recall: 98.53%; FB1: 99.26 67
WDT: precision: 100.00%; recall: 100.00%; FB1: 100.00 19
WRB: precision: 100.00%; recall: 100.00%; FB1: 100.00 2
processed 233 tokens with 233 phrases; found: 233 phrases; correct: 212.
accuracy: 90.99%; precision: 90.99%; recall: 90.99%; FB1: 90.99
(: precision: 100.00%; recall: 100.00%; FB1: 100.00 4
): precision: 80.00%; recall: 100.00%; FB1: 88.89 5
,: precision: 100.00%; recall: 100.00%; FB1: 100.00 7
.: precision: 100.00%; recall: 100.00%; FB1: 100.00 10
CC: precision: 100.00%; recall: 100.00%; FB1: 100.00 9
DT: precision: 100.00%; recall: 100.00%; FB1: 100.00 16
IN: precision: 100.00%; recall: 100.00%; FB1: 100.00 29
JJ: precision: 90.00%; recall: 69.23%; FB1: 78.26 20
NN: precision: 83.54%; recall: 97.06%; FB1: 89.80 79
NNS: precision: 100.00%; recall: 72.22%; FB1: 83.87 13
PRP: precision: 100.00%; recall: 100.00%; FB1: 100.00 2
PRP$: precision: 0.00%; recall: 0.00%; FB1: 0.00 0
RB: precision: 100.00%; recall: 75.00%; FB1: 85.71 3
RP: precision: 0.00%; recall: 0.00%; FB1: 0.00 0
TO: precision: 100.00%; recall: 100.00%; FB1: 100.00 6
VB: precision: 50.00%; recall: 100.00%; FB1: 66.67 2
VBD: precision: 100.00%; recall: 75.00%; FB1: 85.71 3
VBG: precision: 100.00%; recall: 100.00%; FB1: 100.00 5
VBN: precision: 66.67%; recall: 100.00%; FB1: 80.00 12
VBP: precision: 100.00%; recall: 75.00%; FB1: 85.71 3
VBZ: precision: 100.00%; recall: 75.00%; FB1: 85.71 3
WDT: precision: 100.00%; recall: 100.00%; FB1: 100.00 2
4. Using a trained part-of-speech tagger¶
NB: Before proceeding, we have to unzip the trained_models directory so that we can explore and assess the trained model.
In this section, we demonstrate how to revive a trained model for part-of-speech tagging and use it to decode a file comprising new/unseen sequences (i.e. test sequences).
As a reminder, the trained models (including their components) are found under trained_models folder in the cloned repository. We have to unzip them first before proceeding.
To use/revive a trained model dumped on disk, we use revive_learnedmodel(args) function. It takes the path to the trained models' directory.
In [7]:
# we get the trained model parts directory -- check the tree path in the cell above
trained_model_dir = os.path.join(project_dir, 'trained_models', '2017_1_8-0_9_48_894305')
# loading the trained model
crf_m = revive_learnedmodel(trained_model_dir)
After we revive our model, we will use a test dataset that we did not use for model building or training. The test file is found under eval_model folder. The tree path of the directory:
├── 2017_1_8-0_9_48_894305 │ ├── eval_model │ │ ├── test.txt │ │ ├── train.txt │ │ ├── dec_testseqs_fold_0_eval.txt │ │ ├── dec_trainseqs_fold_0_eval.txt
The test file in eval_model folder is composed of multiple sequences that are separated by a newline. An excerpt of the sequences in the test.txt file is provided below:
The DT core NN GGAAs NNS of IN the DT muA NN and CC muB NN sites NNS are VBP separated VBN by IN nucleotides NNS , , suggesting VBG that IN ETS NNS proteins NNS bind VBP to TO these DT sites NNS from IN these DT same JJ side NN of IN the DT DNA NN helix NN . . Our PRP$ results NNS support VBP the DT idea NN that IN human JJ CD36 NN mediates VBZ signal NN transduction NN events NNS in IN response NN to TO Ox NN LDL NN . . In IN contrast NN , , mutation NN of IN the DT kappa NN B NN enhancer NN results VBZ in IN markedly RB attenuated VBN activities NNS of IN both DT promoters NNS . . [ ( Cortisone-resistant JJ bronchial JJ asthma NN ] ) There EX is VBZ general JJ agreement NN on IN the DT inflammatory JJ pathogenesis NN of IN bronchial JJ asthma NN : : an DT accumulation NN of IN activated VBN eosinophils NNS , , degranulated VBN mast NN cells NNS , , T NN lymphocytes NNS and CC in IN very RB severe JJ forms NNS , , granulocytes NNS has VBZ constantly RB been VBN found VBN in IN the DT bronchial JJ mucosa NN . .
To read the file, we will use DataFileParser class in the utilities module.
In [8]:
from pyseqlab.utilities import DataFileParser
# initialize a data file parser
dparser = DataFileParser()
# provide the options to parser such as the header info, the separator between words and if the y label is already existing
# main means the header is found in the first line of the file
header = ('w','label')
# y_ref is a boolean indicating if the label to predict is already found in the file
y_ref = True
# spearator between the observations
column_sep = "\t"
seqs = []
test_file = os.path.join(trained_model_dir, 'eval_model', 'test.txt')
for seq in dparser.read_file(test_file, header, y_ref=y_ref, column_sep = column_sep):
seqs.append(seq)
# printing one sequence for display
print(seqs[0])
print("number of parsed sequences is: ", len(seqs))
Y sequence:
['DT', 'NN', 'NNS', 'IN', 'DT', 'NN', 'CC', 'NN', 'NNS', 'VBP', 'VBN', 'IN', 'NNS', ',', 'VBG', 'IN', 'NNS', 'NNS', 'VBP', 'TO', 'DT', 'NNS', 'IN', 'DT', 'JJ', 'NN', 'IN', 'DT', 'NN', 'NN', '.']
X sequence:
{1: {'w': 'The'}, 2: {'w': 'core'}, 3: {'w': 'GGAAs'}, 4: {'w': 'of'}, 5: {'w': 'the'}, 6: {'w': 'muA'}, 7: {'w': 'and'}, 8: {'w': 'muB'}, 9: {'w': 'sites'}, 10: {'w': 'are'}, 11: {'w': 'separated'}, 12: {'w': 'by'}, 13: {'w': 'nucleotides'}, 14: {'w': ','}, 15: {'w': 'suggesting'}, 16: {'w': 'that'}, 17: {'w': 'ETS'}, 18: {'w': 'proteins'}, 19: {'w': 'bind'}, 20: {'w': 'to'}, 21: {'w': 'these'}, 22: {'w': 'sites'}, 23: {'w': 'from'}, 24: {'w': 'these'}, 25: {'w': 'same'}, 26: {'w': 'side'}, 27: {'w': 'of'}, 28: {'w': 'the'}, 29: {'w': 'DNA'}, 30: {'w': 'helix'}, 31: {'w': '.'}}
----------------------------------------
number of parsed sequences is: 1839
4.1 Decoding method¶
Then, we specify the decoding options for our model to use. The main method for decoding is decode_seqs(decoding_method, out_dir, **kwargs) that takes two arguments and multiple keyword arguments.
The obligatory arguments are:
-
decoding_method: string representing the decoding method such as'viterbi' -
output_dir: string, the output directory representing the path where the parsing would take place
For the keyword arguments, the main ones to specify are:
-
seqs: the list of sequences we already parsed/read from the text file we need to label -
file_name: the name of the file where decoded sequences will be written to (it is optional) -
sep: the separator between the columns/observations when writing decoded sequences to the specified file usingfile_namekeyword argument
In [10]:
decoding_method = 'viterbi'
output_dir = os.path.join(project_dir, 'tutorials')
sep = "\t"
# decode sequences
seqs_decoded = crf_m.decode_seqs(decoding_method, output_dir, seqs= seqs,
beam_size=5,
file_name = 'tutorial_seqs_decoding.txt', sep=sep)
identifying model active features -- processed seqs: 1 identifying model active features -- processed seqs: 2 identifying model active features -- processed seqs: 3 identifying model active features -- processed seqs: 4 identifying model active features -- processed seqs: 5 identifying model active features -- processed seqs: 6 identifying model active features -- processed seqs: 7 identifying model active features -- processed seqs: 8 identifying model active features -- processed seqs: 9 identifying model active features -- processed seqs: 10 identifying model active features -- processed seqs: 11 identifying model active features -- processed seqs: 12 identifying model active features -- processed seqs: 13 identifying model active features -- processed seqs: 14 identifying model active features -- processed seqs: 15 identifying model active features -- processed seqs: 16 identifying model active features -- processed seqs: 17 identifying model active features -- processed seqs: 18 identifying model active features -- processed seqs: 19 identifying model active features -- processed seqs: 20 identifying model active features -- processed seqs: 21 identifying model active features -- processed seqs: 22 identifying model active features -- processed seqs: 23 identifying model active features -- processed seqs: 24 identifying model active features -- processed seqs: 25 identifying model active features -- processed seqs: 26 identifying model active features -- processed seqs: 27 identifying model active features -- processed seqs: 28 identifying model active features -- processed seqs: 29 identifying model active features -- processed seqs: 30 identifying model active features -- processed seqs: 31 identifying model active features -- processed seqs: 32 identifying model active features -- processed seqs: 33 identifying model active features -- processed seqs: 34 identifying model active features -- processed seqs: 35 identifying model active features -- processed seqs: 36 identifying model active features -- processed seqs: 37 identifying model active features -- processed seqs: 38 identifying model active features -- processed seqs: 39 identifying model active features -- processed seqs: 40 identifying model active features -- processed seqs: 41 identifying model active features -- processed seqs: 42 identifying model active features -- processed seqs: 43 identifying model active features -- processed seqs: 44 identifying model active features -- processed seqs: 45 identifying model active features -- processed seqs: 46 identifying model active features -- processed seqs: 47 identifying model active features -- processed seqs: 48 identifying model active features -- processed seqs: 49 identifying model active features -- processed seqs: 50 identifying model active features -- processed seqs: 51 identifying model active features -- processed seqs: 52 identifying model active features -- processed seqs: 53 identifying model active features -- processed seqs: 54 identifying model active features -- processed seqs: 55 identifying model active features -- processed seqs: 56 identifying model active features -- processed seqs: 57 identifying model active features -- processed seqs: 58 identifying model active features -- processed seqs: 59 identifying model active features -- processed seqs: 60 identifying model active features -- processed seqs: 61 identifying model active features -- processed seqs: 62 identifying model active features -- processed seqs: 63 identifying model active features -- processed seqs: 64 identifying model active features -- processed seqs: 65 identifying model active features -- processed seqs: 66 identifying model active features -- processed seqs: 67 identifying model active features -- processed seqs: 68 identifying model active features -- processed seqs: 69 identifying model active features -- processed seqs: 70 identifying model active features -- processed seqs: 71 identifying model active features -- processed seqs: 72 identifying model active features -- processed seqs: 73 identifying model active features -- processed seqs: 74 identifying model active features -- processed seqs: 75 identifying model active features -- processed seqs: 76 identifying model active features -- processed seqs: 77 identifying model active features -- processed seqs: 78 identifying model active features -- processed seqs: 79 identifying model active features -- processed seqs: 80 identifying model active features -- processed seqs: 81 identifying model active features -- processed seqs: 82 identifying model active features -- processed seqs: 83 identifying model active features -- processed seqs: 84 identifying model active features -- processed seqs: 85 identifying model active features -- processed seqs: 86 identifying model active features -- processed seqs: 87 identifying model active features -- processed seqs: 88 identifying model active features -- processed seqs: 89 identifying model active features -- processed seqs: 90 identifying model active features -- processed seqs: 91 identifying model active features -- processed seqs: 92 identifying model active features -- processed seqs: 93 identifying model active features -- processed seqs: 94 identifying model active features -- processed seqs: 95 identifying model active features -- processed seqs: 96 identifying model active features -- processed seqs: 97 identifying model active features -- processed seqs: 98 identifying model active features -- processed seqs: 99 identifying model active features -- processed seqs: 100 identifying model active features -- processed seqs: 101 identifying model active features -- processed seqs: 102 identifying model active features -- processed seqs: 103 identifying model active features -- processed seqs: 104 identifying model active features -- processed seqs: 105 identifying model active features -- processed seqs: 106 identifying model active features -- processed seqs: 107 identifying model active features -- processed seqs: 108 identifying model active features -- processed seqs: 109 identifying model active features -- processed seqs: 110 identifying model active features -- processed seqs: 111 identifying model active features -- processed seqs: 112 identifying model active features -- processed seqs: 113 identifying model active features -- processed seqs: 114 identifying model active features -- processed seqs: 115 identifying model active features -- processed seqs: 116 identifying model active features -- processed seqs: 117 identifying model active features -- processed seqs: 118 identifying model active features -- processed seqs: 119 identifying model active features -- processed seqs: 120 identifying model active features -- processed seqs: 121 identifying model active features -- processed seqs: 122 identifying model active features -- processed seqs: 123 identifying model active features -- processed seqs: 124 identifying model active features -- processed seqs: 125 identifying model active features -- processed seqs: 126 identifying model active features -- processed seqs: 127 identifying model active features -- processed seqs: 128 identifying model active features -- processed seqs: 129 identifying model active features -- processed seqs: 130 identifying model active features -- processed seqs: 131 identifying model active features -- processed seqs: 132 identifying model active features -- processed seqs: 133 identifying model active features -- processed seqs: 134 identifying model active features -- processed seqs: 135 identifying model active features -- processed seqs: 136 identifying model active features -- processed seqs: 137 identifying model active features -- processed seqs: 138 identifying model active features -- processed seqs: 139 identifying model active features -- processed seqs: 140 identifying model active features -- processed seqs: 141 identifying model active features -- processed seqs: 142 identifying model active features -- processed seqs: 143 identifying model active features -- processed seqs: 144 identifying model active features -- processed seqs: 145 identifying model active features -- processed seqs: 146 identifying model active features -- processed seqs: 147 identifying model active features -- processed seqs: 148 identifying model active features -- processed seqs: 149 identifying model active features -- processed seqs: 150 identifying model active features -- processed seqs: 151 identifying model active features -- processed seqs: 152 identifying model active features -- processed seqs: 153 identifying model active features -- processed seqs: 154 identifying model active features -- processed seqs: 155 identifying model active features -- processed seqs: 156 identifying model active features -- processed seqs: 157 identifying model active features -- processed seqs: 158 identifying model active features -- processed seqs: 159 identifying model active features -- processed seqs: 160 identifying model active features -- processed seqs: 161 identifying model active features -- processed seqs: 162 identifying model active features -- processed seqs: 163 identifying model active features -- processed seqs: 164 identifying model active features -- processed seqs: 165 identifying model active features -- processed seqs: 166 identifying model active features -- processed seqs: 167 identifying model active features -- processed seqs: 168 identifying model active features -- processed seqs: 169 identifying model active features -- processed seqs: 170 identifying model active features -- processed seqs: 171 identifying model active features -- processed seqs: 172 identifying model active features -- processed seqs: 173 identifying model active features -- processed seqs: 174 identifying model active features -- processed seqs: 175 identifying model active features -- processed seqs: 176 identifying model active features -- processed seqs: 177 identifying model active features -- processed seqs: 178 identifying model active features -- processed seqs: 179 identifying model active features -- processed seqs: 180 identifying model active features -- processed seqs: 181 identifying model active features -- processed seqs: 182 identifying model active features -- processed seqs: 183 identifying model active features -- processed seqs: 184 identifying model active features -- processed seqs: 185 identifying model active features -- processed seqs: 186 identifying model active features -- processed seqs: 187 identifying model active features -- processed seqs: 188 identifying model active features -- processed seqs: 189 identifying model active features -- processed seqs: 190 identifying model active features -- processed seqs: 191 identifying model active features -- processed seqs: 192 identifying model active features -- processed seqs: 193 identifying model active features -- processed seqs: 194 identifying model active features -- processed seqs: 195 identifying model active features -- processed seqs: 196 identifying model active features -- processed seqs: 197 identifying model active features -- processed seqs: 198 identifying model active features -- processed seqs: 199 identifying model active features -- processed seqs: 200 identifying model active features -- processed seqs: 201 identifying model active features -- processed seqs: 202 identifying model active features -- processed seqs: 203 identifying model active features -- processed seqs: 204 identifying model active features -- processed seqs: 205 identifying model active features -- processed seqs: 206 identifying model active features -- processed seqs: 207 identifying model active features -- processed seqs: 208 identifying model active features -- processed seqs: 209 identifying model active features -- processed seqs: 210 identifying model active features -- processed seqs: 211 identifying model active features -- processed seqs: 212 identifying model active features -- processed seqs: 213 identifying model active features -- processed seqs: 214 identifying model active features -- processed seqs: 215 identifying model active features -- processed seqs: 216 identifying model active features -- processed seqs: 217 identifying model active features -- processed seqs: 218 identifying model active features -- processed seqs: 219 identifying model active features -- processed seqs: 220 identifying model active features -- processed seqs: 221 identifying model active features -- processed seqs: 222 identifying model active features -- processed seqs: 223 identifying model active features -- processed seqs: 224 identifying model active features -- processed seqs: 225 identifying model active features -- processed seqs: 226 identifying model active features -- processed seqs: 227 identifying model active features -- processed seqs: 228 identifying model active features -- processed seqs: 229 identifying model active features -- processed seqs: 230 identifying model active features -- processed seqs: 231 identifying model active features -- processed seqs: 232 identifying model active features -- processed seqs: 233 identifying model active features -- processed seqs: 234 identifying model active features -- processed seqs: 235 identifying model active features -- processed seqs: 236 identifying model active features -- processed seqs: 237 identifying model active features -- processed seqs: 238 identifying model active features -- processed seqs: 239 identifying model active features -- processed seqs: 240 identifying model active features -- processed seqs: 241 identifying model active features -- processed seqs: 242 identifying model active features -- processed seqs: 243 identifying model active features -- processed seqs: 244 identifying model active features -- processed seqs: 245 identifying model active features -- processed seqs: 246 identifying model active features -- processed seqs: 247 identifying model active features -- processed seqs: 248 identifying model active features -- processed seqs: 249 identifying model active features -- processed seqs: 250 identifying model active features -- processed seqs: 251 identifying model active features -- processed seqs: 252 identifying model active features -- processed seqs: 253 identifying model active features -- processed seqs: 254 identifying model active features -- processed seqs: 255 identifying model active features -- processed seqs: 256 identifying model active features -- processed seqs: 257 identifying model active features -- processed seqs: 258 identifying model active features -- processed seqs: 259 identifying model active features -- processed seqs: 260 identifying model active features -- processed seqs: 261 identifying model active features -- processed seqs: 262 identifying model active features -- processed seqs: 263 identifying model active features -- processed seqs: 264 identifying model active features -- processed seqs: 265 identifying model active features -- processed seqs: 266 identifying model active features -- processed seqs: 267 identifying model active features -- processed seqs: 268 identifying model active features -- processed seqs: 269 identifying model active features -- processed seqs: 270 identifying model active features -- processed seqs: 271 identifying model active features -- processed seqs: 272 identifying model active features -- processed seqs: 273 identifying model active features -- processed seqs: 274 identifying model active features -- processed seqs: 275 identifying model active features -- processed seqs: 276 identifying model active features -- processed seqs: 277 identifying model active features -- processed seqs: 278 identifying model active features -- processed seqs: 279 identifying model active features -- processed seqs: 280 identifying model active features -- processed seqs: 281 identifying model active features -- processed seqs: 282 identifying model active features -- processed seqs: 283 identifying model active features -- processed seqs: 284 identifying model active features -- processed seqs: 285 identifying model active features -- processed seqs: 286 identifying model active features -- processed seqs: 287 identifying model active features -- processed seqs: 288 identifying model active features -- processed seqs: 289 identifying model active features -- processed seqs: 290 identifying model active features -- processed seqs: 291 identifying model active features -- processed seqs: 292 identifying model active features -- processed seqs: 293 identifying model active features -- processed seqs: 294 identifying model active features -- processed seqs: 295 identifying model active features -- processed seqs: 296 identifying model active features -- processed seqs: 297 identifying model active features -- processed seqs: 298 identifying model active features -- processed seqs: 299 identifying model active features -- processed seqs: 300 identifying model active features -- processed seqs: 301 identifying model active features -- processed seqs: 302 identifying model active features -- processed seqs: 303 identifying model active features -- processed seqs: 304 identifying model active features -- processed seqs: 305 identifying model active features -- processed seqs: 306 identifying model active features -- processed seqs: 307 identifying model active features -- processed seqs: 308 identifying model active features -- processed seqs: 309 identifying model active features -- processed seqs: 310 identifying model active features -- processed seqs: 311 identifying model active features -- processed seqs: 312 identifying model active features -- processed seqs: 313 identifying model active features -- processed seqs: 314 identifying model active features -- processed seqs: 315 identifying model active features -- processed seqs: 316 identifying model active features -- processed seqs: 317 identifying model active features -- processed seqs: 318 identifying model active features -- processed seqs: 319 identifying model active features -- processed seqs: 320 identifying model active features -- processed seqs: 321 identifying model active features -- processed seqs: 322 identifying model active features -- processed seqs: 323 identifying model active features -- processed seqs: 324 identifying model active features -- processed seqs: 325 identifying model active features -- processed seqs: 326 identifying model active features -- processed seqs: 327 identifying model active features -- processed seqs: 328 identifying model active features -- processed seqs: 329 identifying model active features -- processed seqs: 330 identifying model active features -- processed seqs: 331 identifying model active features -- processed seqs: 332 identifying model active features -- processed seqs: 333 identifying model active features -- processed seqs: 334 identifying model active features -- processed seqs: 335 identifying model active features -- processed seqs: 336 identifying model active features -- processed seqs: 337 identifying model active features -- processed seqs: 338 identifying model active features -- processed seqs: 339 identifying model active features -- processed seqs: 340 identifying model active features -- processed seqs: 341 identifying model active features -- processed seqs: 342 identifying model active features -- processed seqs: 343 identifying model active features -- processed seqs: 344 identifying model active features -- processed seqs: 345 identifying model active features -- processed seqs: 346 identifying model active features -- processed seqs: 347 identifying model active features -- processed seqs: 348 identifying model active features -- processed seqs: 349 identifying model active features -- processed seqs: 350 identifying model active features -- processed seqs: 351 identifying model active features -- processed seqs: 352 identifying model active features -- processed seqs: 353 identifying model active features -- processed seqs: 354 identifying model active features -- processed seqs: 355 identifying model active features -- processed seqs: 356 identifying model active features -- processed seqs: 357 identifying model active features -- processed seqs: 358 identifying model active features -- processed seqs: 359 identifying model active features -- processed seqs: 360 identifying model active features -- processed seqs: 361 identifying model active features -- processed seqs: 362 identifying model active features -- processed seqs: 363 identifying model active features -- processed seqs: 364 identifying model active features -- processed seqs: 365 identifying model active features -- processed seqs: 366 identifying model active features -- processed seqs: 367 identifying model active features -- processed seqs: 368 identifying model active features -- processed seqs: 369 identifying model active features -- processed seqs: 370 identifying model active features -- processed seqs: 371 identifying model active features -- processed seqs: 372 identifying model active features -- processed seqs: 373 identifying model active features -- processed seqs: 374 identifying model active features -- processed seqs: 375 identifying model active features -- processed seqs: 376 identifying model active features -- processed seqs: 377 identifying model active features -- processed seqs: 378 identifying model active features -- processed seqs: 379 identifying model active features -- processed seqs: 380 identifying model active features -- processed seqs: 381 identifying model active features -- processed seqs: 382 identifying model active features -- processed seqs: 383 identifying model active features -- processed seqs: 384 identifying model active features -- processed seqs: 385 identifying model active features -- processed seqs: 386 identifying model active features -- processed seqs: 387 identifying model active features -- processed seqs: 388 identifying model active features -- processed seqs: 389 identifying model active features -- processed seqs: 390 identifying model active features -- processed seqs: 391 identifying model active features -- processed seqs: 392 identifying model active features -- processed seqs: 393 identifying model active features -- processed seqs: 394 identifying model active features -- processed seqs: 395 identifying model active features -- processed seqs: 396 identifying model active features -- processed seqs: 397 identifying model active features -- processed seqs: 398 identifying model active features -- processed seqs: 399 identifying model active features -- processed seqs: 400 identifying model active features -- processed seqs: 401 identifying model active features -- processed seqs: 402 identifying model active features -- processed seqs: 403 identifying model active features -- processed seqs: 404 identifying model active features -- processed seqs: 405 identifying model active features -- processed seqs: 406 identifying model active features -- processed seqs: 407 identifying model active features -- processed seqs: 408 identifying model active features -- processed seqs: 409 identifying model active features -- processed seqs: 410 identifying model active features -- processed seqs: 411 identifying model active features -- processed seqs: 412 identifying model active features -- processed seqs: 413 identifying model active features -- processed seqs: 414 identifying model active features -- processed seqs: 415 identifying model active features -- processed seqs: 416 identifying model active features -- processed seqs: 417 identifying model active features -- processed seqs: 418 identifying model active features -- processed seqs: 419 identifying model active features -- processed seqs: 420 identifying model active features -- processed seqs: 421 identifying model active features -- processed seqs: 422 identifying model active features -- processed seqs: 423 identifying model active features -- processed seqs: 424 identifying model active features -- processed seqs: 425 identifying model active features -- processed seqs: 426 identifying model active features -- processed seqs: 427 identifying model active features -- processed seqs: 428 identifying model active features -- processed seqs: 429 identifying model active features -- processed seqs: 430 identifying model active features -- processed seqs: 431 identifying model active features -- processed seqs: 432 identifying model active features -- processed seqs: 433 identifying model active features -- processed seqs: 434 identifying model active features -- processed seqs: 435 identifying model active features -- processed seqs: 436 identifying model active features -- processed seqs: 437 identifying model active features -- processed seqs: 438 identifying model active features -- processed seqs: 439 identifying model active features -- processed seqs: 440 identifying model active features -- processed seqs: 441 identifying model active features -- processed seqs: 442 identifying model active features -- processed seqs: 443 identifying model active features -- processed seqs: 444 identifying model active features -- processed seqs: 445 identifying model active features -- processed seqs: 446 identifying model active features -- processed seqs: 447 identifying model active features -- processed seqs: 448 identifying model active features -- processed seqs: 449 identifying model active features -- processed seqs: 450 identifying model active features -- processed seqs: 451 identifying model active features -- processed seqs: 452 identifying model active features -- processed seqs: 453 identifying model active features -- processed seqs: 454 identifying model active features -- processed seqs: 455 identifying model active features -- processed seqs: 456 identifying model active features -- processed seqs: 457 identifying model active features -- processed seqs: 458 identifying model active features -- processed seqs: 459 identifying model active features -- processed seqs: 460 identifying model active features -- processed seqs: 461 identifying model active features -- processed seqs: 462 identifying model active features -- processed seqs: 463 identifying model active features -- processed seqs: 464 identifying model active features -- processed seqs: 465 identifying model active features -- processed seqs: 466 identifying model active features -- processed seqs: 467 identifying model active features -- processed seqs: 468 identifying model active features -- processed seqs: 469 identifying model active features -- processed seqs: 470 identifying model active features -- processed seqs: 471 identifying model active features -- processed seqs: 472 identifying model active features -- processed seqs: 473 identifying model active features -- processed seqs: 474 identifying model active features -- processed seqs: 475 identifying model active features -- processed seqs: 476 identifying model active features -- processed seqs: 477 identifying model active features -- processed seqs: 478 identifying model active features -- processed seqs: 479 identifying model active features -- processed seqs: 480 identifying model active features -- processed seqs: 481 identifying model active features -- processed seqs: 482 identifying model active features -- processed seqs: 483 identifying model active features -- processed seqs: 484 identifying model active features -- processed seqs: 485 identifying model active features -- processed seqs: 486 identifying model active features -- processed seqs: 487 identifying model active features -- processed seqs: 488 identifying model active features -- processed seqs: 489 identifying model active features -- processed seqs: 490 identifying model active features -- processed seqs: 491 identifying model active features -- processed seqs: 492 identifying model active features -- processed seqs: 493 identifying model active features -- processed seqs: 494 identifying model active features -- processed seqs: 495 identifying model active features -- processed seqs: 496 identifying model active features -- processed seqs: 497 identifying model active features -- processed seqs: 498 identifying model active features -- processed seqs: 499 identifying model active features -- processed seqs: 500 identifying model active features -- processed seqs: 501 identifying model active features -- processed seqs: 502 identifying model active features -- processed seqs: 503 identifying model active features -- processed seqs: 504 identifying model active features -- processed seqs: 505 identifying model active features -- processed seqs: 506 identifying model active features -- processed seqs: 507 identifying model active features -- processed seqs: 508 identifying model active features -- processed seqs: 509 identifying model active features -- processed seqs: 510 identifying model active features -- processed seqs: 511 identifying model active features -- processed seqs: 512 identifying model active features -- processed seqs: 513 identifying model active features -- processed seqs: 514 identifying model active features -- processed seqs: 515 identifying model active features -- processed seqs: 516 identifying model active features -- processed seqs: 517 identifying model active features -- processed seqs: 518 identifying model active features -- processed seqs: 519 identifying model active features -- processed seqs: 520 identifying model active features -- processed seqs: 521 identifying model active features -- processed seqs: 522 identifying model active features -- processed seqs: 523 identifying model active features -- processed seqs: 524 identifying model active features -- processed seqs: 525 identifying model active features -- processed seqs: 526 identifying model active features -- processed seqs: 527 identifying model active features -- processed seqs: 528 identifying model active features -- processed seqs: 529 identifying model active features -- processed seqs: 530 identifying model active features -- processed seqs: 531 identifying model active features -- processed seqs: 532 identifying model active features -- processed seqs: 533 identifying model active features -- processed seqs: 534 identifying model active features -- processed seqs: 535 identifying model active features -- processed seqs: 536 identifying model active features -- processed seqs: 537 identifying model active features -- processed seqs: 538 identifying model active features -- processed seqs: 539 identifying model active features -- processed seqs: 540 identifying model active features -- processed seqs: 541 identifying model active features -- processed seqs: 542 identifying model active features -- processed seqs: 543 identifying model active features -- processed seqs: 544 identifying model active features -- processed seqs: 545 identifying model active features -- processed seqs: 546 identifying model active features -- processed seqs: 547 identifying model active features -- processed seqs: 548 identifying model active features -- processed seqs: 549 identifying model active features -- processed seqs: 550 identifying model active features -- processed seqs: 551 identifying model active features -- processed seqs: 552 identifying model active features -- processed seqs: 553 identifying model active features -- processed seqs: 554 identifying model active features -- processed seqs: 555 identifying model active features -- processed seqs: 556 identifying model active features -- processed seqs: 557 identifying model active features -- processed seqs: 558 identifying model active features -- processed seqs: 559 identifying model active features -- processed seqs: 560 identifying model active features -- processed seqs: 561 identifying model active features -- processed seqs: 562 identifying model active features -- processed seqs: 563 identifying model active features -- processed seqs: 564 identifying model active features -- processed seqs: 565 identifying model active features -- processed seqs: 566 identifying model active features -- processed seqs: 567 identifying model active features -- processed seqs: 568 identifying model active features -- processed seqs: 569 identifying model active features -- processed seqs: 570 identifying model active features -- processed seqs: 571 identifying model active features -- processed seqs: 572 identifying model active features -- processed seqs: 573 identifying model active features -- processed seqs: 574 identifying model active features -- processed seqs: 575 identifying model active features -- processed seqs: 576 identifying model active features -- processed seqs: 577 identifying model active features -- processed seqs: 578 identifying model active features -- processed seqs: 579 identifying model active features -- processed seqs: 580 identifying model active features -- processed seqs: 581 identifying model active features -- processed seqs: 582 identifying model active features -- processed seqs: 583 identifying model active features -- processed seqs: 584 identifying model active features -- processed seqs: 585 identifying model active features -- processed seqs: 586 identifying model active features -- processed seqs: 587 identifying model active features -- processed seqs: 588 identifying model active features -- processed seqs: 589 identifying model active features -- processed seqs: 590 identifying model active features -- processed seqs: 591 identifying model active features -- processed seqs: 592 identifying model active features -- processed seqs: 593 identifying model active features -- processed seqs: 594 identifying model active features -- processed seqs: 595 identifying model active features -- processed seqs: 596 identifying model active features -- processed seqs: 597 identifying model active features -- processed seqs: 598 identifying model active features -- processed seqs: 599 identifying model active features -- processed seqs: 600 identifying model active features -- processed seqs: 601 identifying model active features -- processed seqs: 602 identifying model active features -- processed seqs: 603 identifying model active features -- processed seqs: 604 identifying model active features -- processed seqs: 605 identifying model active features -- processed seqs: 606 identifying model active features -- processed seqs: 607 identifying model active features -- processed seqs: 608 identifying model active features -- processed seqs: 609 identifying model active features -- processed seqs: 610 identifying model active features -- processed seqs: 611 identifying model active features -- processed seqs: 612 identifying model active features -- processed seqs: 613 identifying model active features -- processed seqs: 614 identifying model active features -- processed seqs: 615 identifying model active features -- processed seqs: 616 identifying model active features -- processed seqs: 617 identifying model active features -- processed seqs: 618 identifying model active features -- processed seqs: 619 identifying model active features -- processed seqs: 620 identifying model active features -- processed seqs: 621 identifying model active features -- processed seqs: 622 identifying model active features -- processed seqs: 623 identifying model active features -- processed seqs: 624 identifying model active features -- processed seqs: 625 identifying model active features -- processed seqs: 626 identifying model active features -- processed seqs: 627 identifying model active features -- processed seqs: 628 identifying model active features -- processed seqs: 629 identifying model active features -- processed seqs: 630 identifying model active features -- processed seqs: 631 identifying model active features -- processed seqs: 632 identifying model active features -- processed seqs: 633 identifying model active features -- processed seqs: 634 identifying model active features -- processed seqs: 635 identifying model active features -- processed seqs: 636 identifying model active features -- processed seqs: 637 identifying model active features -- processed seqs: 638 identifying model active features -- processed seqs: 639 identifying model active features -- processed seqs: 640 identifying model active features -- processed seqs: 641 identifying model active features -- processed seqs: 642 identifying model active features -- processed seqs: 643 identifying model active features -- processed seqs: 644 identifying model active features -- processed seqs: 645 identifying model active features -- processed seqs: 646 identifying model active features -- processed seqs: 647 identifying model active features -- processed seqs: 648 identifying model active features -- processed seqs: 649 identifying model active features -- processed seqs: 650 identifying model active features -- processed seqs: 651 identifying model active features -- processed seqs: 652 identifying model active features -- processed seqs: 653 identifying model active features -- processed seqs: 654 identifying model active features -- processed seqs: 655 identifying model active features -- processed seqs: 656 identifying model active features -- processed seqs: 657 identifying model active features -- processed seqs: 658 identifying model active features -- processed seqs: 659 identifying model active features -- processed seqs: 660 identifying model active features -- processed seqs: 661 identifying model active features -- processed seqs: 662 identifying model active features -- processed seqs: 663 identifying model active features -- processed seqs: 664 identifying model active features -- processed seqs: 665 identifying model active features -- processed seqs: 666 identifying model active features -- processed seqs: 667 identifying model active features -- processed seqs: 668 identifying model active features -- processed seqs: 669 identifying model active features -- processed seqs: 670 identifying model active features -- processed seqs: 671 identifying model active features -- processed seqs: 672 identifying model active features -- processed seqs: 673 identifying model active features -- processed seqs: 674 identifying model active features -- processed seqs: 675 identifying model active features -- processed seqs: 676 identifying model active features -- processed seqs: 677 identifying model active features -- processed seqs: 678 identifying model active features -- processed seqs: 679 identifying model active features -- processed seqs: 680 identifying model active features -- processed seqs: 681 identifying model active features -- processed seqs: 682 identifying model active features -- processed seqs: 683 identifying model active features -- processed seqs: 684 identifying model active features -- processed seqs: 685 identifying model active features -- processed seqs: 686 identifying model active features -- processed seqs: 687 identifying model active features -- processed seqs: 688 identifying model active features -- processed seqs: 689 identifying model active features -- processed seqs: 690 identifying model active features -- processed seqs: 691 identifying model active features -- processed seqs: 692 identifying model active features -- processed seqs: 693 identifying model active features -- processed seqs: 694 identifying model active features -- processed seqs: 695 identifying model active features -- processed seqs: 696 identifying model active features -- processed seqs: 697 identifying model active features -- processed seqs: 698 identifying model active features -- processed seqs: 699 identifying model active features -- processed seqs: 700 identifying model active features -- processed seqs: 701 identifying model active features -- processed seqs: 702 identifying model active features -- processed seqs: 703 identifying model active features -- processed seqs: 704 identifying model active features -- processed seqs: 705 identifying model active features -- processed seqs: 706 identifying model active features -- processed seqs: 707 identifying model active features -- processed seqs: 708 identifying model active features -- processed seqs: 709 identifying model active features -- processed seqs: 710 identifying model active features -- processed seqs: 711 identifying model active features -- processed seqs: 712 identifying model active features -- processed seqs: 713 identifying model active features -- processed seqs: 714 identifying model active features -- processed seqs: 715 identifying model active features -- processed seqs: 716 identifying model active features -- processed seqs: 717 identifying model active features -- processed seqs: 718 identifying model active features -- processed seqs: 719 identifying model active features -- processed seqs: 720 identifying model active features -- processed seqs: 721 identifying model active features -- processed seqs: 722 identifying model active features -- processed seqs: 723 identifying model active features -- processed seqs: 724 identifying model active features -- processed seqs: 725 identifying model active features -- processed seqs: 726 identifying model active features -- processed seqs: 727 identifying model active features -- processed seqs: 728 identifying model active features -- processed seqs: 729 identifying model active features -- processed seqs: 730 identifying model active features -- processed seqs: 731 identifying model active features -- processed seqs: 732 identifying model active features -- processed seqs: 733 identifying model active features -- processed seqs: 734 identifying model active features -- processed seqs: 735 identifying model active features -- processed seqs: 736 identifying model active features -- processed seqs: 737 identifying model active features -- processed seqs: 738 identifying model active features -- processed seqs: 739 identifying model active features -- processed seqs: 740 identifying model active features -- processed seqs: 741 identifying model active features -- processed seqs: 742 identifying model active features -- processed seqs: 743 identifying model active features -- processed seqs: 744 identifying model active features -- processed seqs: 745 identifying model active features -- processed seqs: 746 identifying model active features -- processed seqs: 747 identifying model active features -- processed seqs: 748 identifying model active features -- processed seqs: 749 identifying model active features -- processed seqs: 750 identifying model active features -- processed seqs: 751 identifying model active features -- processed seqs: 752 identifying model active features -- processed seqs: 753 identifying model active features -- processed seqs: 754 identifying model active features -- processed seqs: 755 identifying model active features -- processed seqs: 756 identifying model active features -- processed seqs: 757 identifying model active features -- processed seqs: 758 identifying model active features -- processed seqs: 759 identifying model active features -- processed seqs: 760 identifying model active features -- processed seqs: 761 identifying model active features -- processed seqs: 762 identifying model active features -- processed seqs: 763 identifying model active features -- processed seqs: 764 identifying model active features -- processed seqs: 765 identifying model active features -- processed seqs: 766 identifying model active features -- processed seqs: 767 identifying model active features -- processed seqs: 768 identifying model active features -- processed seqs: 769 identifying model active features -- processed seqs: 770 identifying model active features -- processed seqs: 771 identifying model active features -- processed seqs: 772 identifying model active features -- processed seqs: 773 identifying model active features -- processed seqs: 774 identifying model active features -- processed seqs: 775 identifying model active features -- processed seqs: 776 identifying model active features -- processed seqs: 777 identifying model active features -- processed seqs: 778 identifying model active features -- processed seqs: 779 identifying model active features -- processed seqs: 780 identifying model active features -- processed seqs: 781 identifying model active features -- processed seqs: 782 identifying model active features -- processed seqs: 783 identifying model active features -- processed seqs: 784 identifying model active features -- processed seqs: 785 identifying model active features -- processed seqs: 786 identifying model active features -- processed seqs: 787 identifying model active features -- processed seqs: 788 identifying model active features -- processed seqs: 789 identifying model active features -- processed seqs: 790 identifying model active features -- processed seqs: 791 identifying model active features -- processed seqs: 792 identifying model active features -- processed seqs: 793 identifying model active features -- processed seqs: 794 identifying model active features -- processed seqs: 795 identifying model active features -- processed seqs: 796 identifying model active features -- processed seqs: 797 identifying model active features -- processed seqs: 798 identifying model active features -- processed seqs: 799 identifying model active features -- processed seqs: 800 identifying model active features -- processed seqs: 801 identifying model active features -- processed seqs: 802 identifying model active features -- processed seqs: 803 identifying model active features -- processed seqs: 804 identifying model active features -- processed seqs: 805 identifying model active features -- processed seqs: 806 identifying model active features -- processed seqs: 807 identifying model active features -- processed seqs: 808 identifying model active features -- processed seqs: 809 identifying model active features -- processed seqs: 810 identifying model active features -- processed seqs: 811 identifying model active features -- processed seqs: 812 identifying model active features -- processed seqs: 813 identifying model active features -- processed seqs: 814 identifying model active features -- processed seqs: 815 identifying model active features -- processed seqs: 816 identifying model active features -- processed seqs: 817 identifying model active features -- processed seqs: 818 identifying model active features -- processed seqs: 819 identifying model active features -- processed seqs: 820 identifying model active features -- processed seqs: 821 identifying model active features -- processed seqs: 822 identifying model active features -- processed seqs: 823 identifying model active features -- processed seqs: 824 identifying model active features -- processed seqs: 825 identifying model active features -- processed seqs: 826 identifying model active features -- processed seqs: 827 identifying model active features -- processed seqs: 828 identifying model active features -- processed seqs: 829 identifying model active features -- processed seqs: 830 identifying model active features -- processed seqs: 831 identifying model active features -- processed seqs: 832 identifying model active features -- processed seqs: 833 identifying model active features -- processed seqs: 834 identifying model active features -- processed seqs: 835 identifying model active features -- processed seqs: 836 identifying model active features -- processed seqs: 837 identifying model active features -- processed seqs: 838 identifying model active features -- processed seqs: 839 identifying model active features -- processed seqs: 840 identifying model active features -- processed seqs: 841 identifying model active features -- processed seqs: 842 identifying model active features -- processed seqs: 843 identifying model active features -- processed seqs: 844 identifying model active features -- processed seqs: 845 identifying model active features -- processed seqs: 846 identifying model active features -- processed seqs: 847 identifying model active features -- processed seqs: 848 identifying model active features -- processed seqs: 849 identifying model active features -- processed seqs: 850 identifying model active features -- processed seqs: 851 identifying model active features -- processed seqs: 852 identifying model active features -- processed seqs: 853 identifying model active features -- processed seqs: 854 identifying model active features -- processed seqs: 855 identifying model active features -- processed seqs: 856 identifying model active features -- processed seqs: 857 identifying model active features -- processed seqs: 858 identifying model active features -- processed seqs: 859 identifying model active features -- processed seqs: 860 identifying model active features -- processed seqs: 861 identifying model active features -- processed seqs: 862 identifying model active features -- processed seqs: 863 identifying model active features -- processed seqs: 864 identifying model active features -- processed seqs: 865 identifying model active features -- processed seqs: 866 identifying model active features -- processed seqs: 867 identifying model active features -- processed seqs: 868 identifying model active features -- processed seqs: 869 identifying model active features -- processed seqs: 870 identifying model active features -- processed seqs: 871 identifying model active features -- processed seqs: 872 identifying model active features -- processed seqs: 873 identifying model active features -- processed seqs: 874 identifying model active features -- processed seqs: 875 identifying model active features -- processed seqs: 876 identifying model active features -- processed seqs: 877 identifying model active features -- processed seqs: 878 identifying model active features -- processed seqs: 879 identifying model active features -- processed seqs: 880 identifying model active features -- processed seqs: 881 identifying model active features -- processed seqs: 882 identifying model active features -- processed seqs: 883 identifying model active features -- processed seqs: 884 identifying model active features -- processed seqs: 885 identifying model active features -- processed seqs: 886 identifying model active features -- processed seqs: 887 identifying model active features -- processed seqs: 888 identifying model active features -- processed seqs: 889 identifying model active features -- processed seqs: 890 identifying model active features -- processed seqs: 891 identifying model active features -- processed seqs: 892 identifying model active features -- processed seqs: 893 identifying model active features -- processed seqs: 894 identifying model active features -- processed seqs: 895 identifying model active features -- processed seqs: 896 identifying model active features -- processed seqs: 897 identifying model active features -- processed seqs: 898 identifying model active features -- processed seqs: 899 identifying model active features -- processed seqs: 900 identifying model active features -- processed seqs: 901 identifying model active features -- processed seqs: 902 identifying model active features -- processed seqs: 903 identifying model active features -- processed seqs: 904 identifying model active features -- processed seqs: 905 identifying model active features -- processed seqs: 906 identifying model active features -- processed seqs: 907 identifying model active features -- processed seqs: 908 identifying model active features -- processed seqs: 909 identifying model active features -- processed seqs: 910 identifying model active features -- processed seqs: 911 identifying model active features -- processed seqs: 912 identifying model active features -- processed seqs: 913 identifying model active features -- processed seqs: 914 identifying model active features -- processed seqs: 915 identifying model active features -- processed seqs: 916 identifying model active features -- processed seqs: 917 identifying model active features -- processed seqs: 918 identifying model active features -- processed seqs: 919 identifying model active features -- processed seqs: 920 identifying model active features -- processed seqs: 921 identifying model active features -- processed seqs: 922 identifying model active features -- processed seqs: 923 identifying model active features -- processed seqs: 924 identifying model active features -- processed seqs: 925 identifying model active features -- processed seqs: 926 identifying model active features -- processed seqs: 927 identifying model active features -- processed seqs: 928 identifying model active features -- processed seqs: 929 identifying model active features -- processed seqs: 930 identifying model active features -- processed seqs: 931 identifying model active features -- processed seqs: 932 identifying model active features -- processed seqs: 933 identifying model active features -- processed seqs: 934 identifying model active features -- processed seqs: 935 identifying model active features -- processed seqs: 936 identifying model active features -- processed seqs: 937 identifying model active features -- processed seqs: 938 identifying model active features -- processed seqs: 939 identifying model active features -- processed seqs: 940 identifying model active features -- processed seqs: 941 identifying model active features -- processed seqs: 942 identifying model active features -- processed seqs: 943 identifying model active features -- processed seqs: 944 identifying model active features -- processed seqs: 945 identifying model active features -- processed seqs: 946 identifying model active features -- processed seqs: 947 identifying model active features -- processed seqs: 948 identifying model active features -- processed seqs: 949 identifying model active features -- processed seqs: 950 identifying model active features -- processed seqs: 951 identifying model active features -- processed seqs: 952 identifying model active features -- processed seqs: 953 identifying model active features -- processed seqs: 954 identifying model active features -- processed seqs: 955 identifying model active features -- processed seqs: 956 identifying model active features -- processed seqs: 957 identifying model active features -- processed seqs: 958 identifying model active features -- processed seqs: 959 identifying model active features -- processed seqs: 960 identifying model active features -- processed seqs: 961 identifying model active features -- processed seqs: 962 identifying model active features -- processed seqs: 963 identifying model active features -- processed seqs: 964 identifying model active features -- processed seqs: 965 identifying model active features -- processed seqs: 966 identifying model active features -- processed seqs: 967 identifying model active features -- processed seqs: 968 identifying model active features -- processed seqs: 969 identifying model active features -- processed seqs: 970 identifying model active features -- processed seqs: 971 identifying model active features -- processed seqs: 972 identifying model active features -- processed seqs: 973 identifying model active features -- processed seqs: 974 identifying model active features -- processed seqs: 975 identifying model active features -- processed seqs: 976 identifying model active features -- processed seqs: 977 identifying model active features -- processed seqs: 978 identifying model active features -- processed seqs: 979 identifying model active features -- processed seqs: 980 identifying model active features -- processed seqs: 981 identifying model active features -- processed seqs: 982 identifying model active features -- processed seqs: 983 identifying model active features -- processed seqs: 984 identifying model active features -- processed seqs: 985 identifying model active features -- processed seqs: 986 identifying model active features -- processed seqs: 987 identifying model active features -- processed seqs: 988 identifying model active features -- processed seqs: 989 identifying model active features -- processed seqs: 990 identifying model active features -- processed seqs: 991 identifying model active features -- processed seqs: 992 identifying model active features -- processed seqs: 993 identifying model active features -- processed seqs: 994 identifying model active features -- processed seqs: 995 identifying model active features -- processed seqs: 996 identifying model active features -- processed seqs: 997 identifying model active features -- processed seqs: 998 identifying model active features -- processed seqs: 999 identifying model active features -- processed seqs: 1000 identifying model active features -- processed seqs: 1001 identifying model active features -- processed seqs: 1002 identifying model active features -- processed seqs: 1003 identifying model active features -- processed seqs: 1004 identifying model active features -- processed seqs: 1005 identifying model active features -- processed seqs: 1006 identifying model active features -- processed seqs: 1007 identifying model active features -- processed seqs: 1008 identifying model active features -- processed seqs: 1009 identifying model active features -- processed seqs: 1010 identifying model active features -- processed seqs: 1011 identifying model active features -- processed seqs: 1012 identifying model active features -- processed seqs: 1013 identifying model active features -- processed seqs: 1014 identifying model active features -- processed seqs: 1015 identifying model active features -- processed seqs: 1016 identifying model active features -- processed seqs: 1017 identifying model active features -- processed seqs: 1018 identifying model active features -- processed seqs: 1019 identifying model active features -- processed seqs: 1020 identifying model active features -- processed seqs: 1021 identifying model active features -- processed seqs: 1022 identifying model active features -- processed seqs: 1023 identifying model active features -- processed seqs: 1024 identifying model active features -- processed seqs: 1025 identifying model active features -- processed seqs: 1026 identifying model active features -- processed seqs: 1027 identifying model active features -- processed seqs: 1028 identifying model active features -- processed seqs: 1029 identifying model active features -- processed seqs: 1030 identifying model active features -- processed seqs: 1031 identifying model active features -- processed seqs: 1032 identifying model active features -- processed seqs: 1033 identifying model active features -- processed seqs: 1034 identifying model active features -- processed seqs: 1035 identifying model active features -- processed seqs: 1036 identifying model active features -- processed seqs: 1037 identifying model active features -- processed seqs: 1038 identifying model active features -- processed seqs: 1039 identifying model active features -- processed seqs: 1040 identifying model active features -- processed seqs: 1041 identifying model active features -- processed seqs: 1042 identifying model active features -- processed seqs: 1043 identifying model active features -- processed seqs: 1044 identifying model active features -- processed seqs: 1045 identifying model active features -- processed seqs: 1046 identifying model active features -- processed seqs: 1047 identifying model active features -- processed seqs: 1048 identifying model active features -- processed seqs: 1049 identifying model active features -- processed seqs: 1050 identifying model active features -- processed seqs: 1051 identifying model active features -- processed seqs: 1052 identifying model active features -- processed seqs: 1053 identifying model active features -- processed seqs: 1054 identifying model active features -- processed seqs: 1055 identifying model active features -- processed seqs: 1056 identifying model active features -- processed seqs: 1057 identifying model active features -- processed seqs: 1058 identifying model active features -- processed seqs: 1059 identifying model active features -- processed seqs: 1060 identifying model active features -- processed seqs: 1061 identifying model active features -- processed seqs: 1062 identifying model active features -- processed seqs: 1063 identifying model active features -- processed seqs: 1064 identifying model active features -- processed seqs: 1065 identifying model active features -- processed seqs: 1066 identifying model active features -- processed seqs: 1067 identifying model active features -- processed seqs: 1068 identifying model active features -- processed seqs: 1069 identifying model active features -- processed seqs: 1070 identifying model active features -- processed seqs: 1071 identifying model active features -- processed seqs: 1072 identifying model active features -- processed seqs: 1073 identifying model active features -- processed seqs: 1074 identifying model active features -- processed seqs: 1075 identifying model active features -- processed seqs: 1076 identifying model active features -- processed seqs: 1077 identifying model active features -- processed seqs: 1078 identifying model active features -- processed seqs: 1079 identifying model active features -- processed seqs: 1080 identifying model active features -- processed seqs: 1081 identifying model active features -- processed seqs: 1082 identifying model active features -- processed seqs: 1083 identifying model active features -- processed seqs: 1084 identifying model active features -- processed seqs: 1085 identifying model active features -- processed seqs: 1086 identifying model active features -- processed seqs: 1087 identifying model active features -- processed seqs: 1088 identifying model active features -- processed seqs: 1089 identifying model active features -- processed seqs: 1090 identifying model active features -- processed seqs: 1091 identifying model active features -- processed seqs: 1092 identifying model active features -- processed seqs: 1093 identifying model active features -- processed seqs: 1094 identifying model active features -- processed seqs: 1095 identifying model active features -- processed seqs: 1096 identifying model active features -- processed seqs: 1097 identifying model active features -- processed seqs: 1098 identifying model active features -- processed seqs: 1099 identifying model active features -- processed seqs: 1100 identifying model active features -- processed seqs: 1101 identifying model active features -- processed seqs: 1102 identifying model active features -- processed seqs: 1103 identifying model active features -- processed seqs: 1104 identifying model active features -- processed seqs: 1105 identifying model active features -- processed seqs: 1106 identifying model active features -- processed seqs: 1107 identifying model active features -- processed seqs: 1108 identifying model active features -- processed seqs: 1109 identifying model active features -- processed seqs: 1110 identifying model active features -- processed seqs: 1111 identifying model active features -- processed seqs: 1112 identifying model active features -- processed seqs: 1113 identifying model active features -- processed seqs: 1114 identifying model active features -- processed seqs: 1115 identifying model active features -- processed seqs: 1116 identifying model active features -- processed seqs: 1117 identifying model active features -- processed seqs: 1118 identifying model active features -- processed seqs: 1119 identifying model active features -- processed seqs: 1120 identifying model active features -- processed seqs: 1121 identifying model active features -- processed seqs: 1122 identifying model active features -- processed seqs: 1123 identifying model active features -- processed seqs: 1124 identifying model active features -- processed seqs: 1125 identifying model active features -- processed seqs: 1126 identifying model active features -- processed seqs: 1127 identifying model active features -- processed seqs: 1128 identifying model active features -- processed seqs: 1129 identifying model active features -- processed seqs: 1130 identifying model active features -- processed seqs: 1131 identifying model active features -- processed seqs: 1132 identifying model active features -- processed seqs: 1133 identifying model active features -- processed seqs: 1134 identifying model active features -- processed seqs: 1135 identifying model active features -- processed seqs: 1136 identifying model active features -- processed seqs: 1137 identifying model active features -- processed seqs: 1138 identifying model active features -- processed seqs: 1139 identifying model active features -- processed seqs: 1140 identifying model active features -- processed seqs: 1141 identifying model active features -- processed seqs: 1142 identifying model active features -- processed seqs: 1143 identifying model active features -- processed seqs: 1144 identifying model active features -- processed seqs: 1145 identifying model active features -- processed seqs: 1146 identifying model active features -- processed seqs: 1147 identifying model active features -- processed seqs: 1148 identifying model active features -- processed seqs: 1149 identifying model active features -- processed seqs: 1150 identifying model active features -- processed seqs: 1151 identifying model active features -- processed seqs: 1152 identifying model active features -- processed seqs: 1153 identifying model active features -- processed seqs: 1154 identifying model active features -- processed seqs: 1155 identifying model active features -- processed seqs: 1156 identifying model active features -- processed seqs: 1157 identifying model active features -- processed seqs: 1158 identifying model active features -- processed seqs: 1159 identifying model active features -- processed seqs: 1160 identifying model active features -- processed seqs: 1161 identifying model active features -- processed seqs: 1162 identifying model active features -- processed seqs: 1163 identifying model active features -- processed seqs: 1164 identifying model active features -- processed seqs: 1165 identifying model active features -- processed seqs: 1166 identifying model active features -- processed seqs: 1167 identifying model active features -- processed seqs: 1168 identifying model active features -- processed seqs: 1169 identifying model active features -- processed seqs: 1170 identifying model active features -- processed seqs: 1171 identifying model active features -- processed seqs: 1172 identifying model active features -- processed seqs: 1173 identifying model active features -- processed seqs: 1174 identifying model active features -- processed seqs: 1175 identifying model active features -- processed seqs: 1176 identifying model active features -- processed seqs: 1177 identifying model active features -- processed seqs: 1178 identifying model active features -- processed seqs: 1179 identifying model active features -- processed seqs: 1180 identifying model active features -- processed seqs: 1181 identifying model active features -- processed seqs: 1182 identifying model active features -- processed seqs: 1183 identifying model active features -- processed seqs: 1184 identifying model active features -- processed seqs: 1185 identifying model active features -- processed seqs: 1186 identifying model active features -- processed seqs: 1187 identifying model active features -- processed seqs: 1188 identifying model active features -- processed seqs: 1189 identifying model active features -- processed seqs: 1190 identifying model active features -- processed seqs: 1191 identifying model active features -- processed seqs: 1192 identifying model active features -- processed seqs: 1193 identifying model active features -- processed seqs: 1194 identifying model active features -- processed seqs: 1195 identifying model active features -- processed seqs: 1196 identifying model active features -- processed seqs: 1197 identifying model active features -- processed seqs: 1198 identifying model active features -- processed seqs: 1199 identifying model active features -- processed seqs: 1200 identifying model active features -- processed seqs: 1201 identifying model active features -- processed seqs: 1202 identifying model active features -- processed seqs: 1203 identifying model active features -- processed seqs: 1204 identifying model active features -- processed seqs: 1205 identifying model active features -- processed seqs: 1206 identifying model active features -- processed seqs: 1207 identifying model active features -- processed seqs: 1208 identifying model active features -- processed seqs: 1209 identifying model active features -- processed seqs: 1210 identifying model active features -- processed seqs: 1211 identifying model active features -- processed seqs: 1212 identifying model active features -- processed seqs: 1213 identifying model active features -- processed seqs: 1214 identifying model active features -- processed seqs: 1215 identifying model active features -- processed seqs: 1216 identifying model active features -- processed seqs: 1217 identifying model active features -- processed seqs: 1218 identifying model active features -- processed seqs: 1219 identifying model active features -- processed seqs: 1220 identifying model active features -- processed seqs: 1221 identifying model active features -- processed seqs: 1222 identifying model active features -- processed seqs: 1223 identifying model active features -- processed seqs: 1224 identifying model active features -- processed seqs: 1225 identifying model active features -- processed seqs: 1226 identifying model active features -- processed seqs: 1227 identifying model active features -- processed seqs: 1228 identifying model active features -- processed seqs: 1229 identifying model active features -- processed seqs: 1230 identifying model active features -- processed seqs: 1231 identifying model active features -- processed seqs: 1232 identifying model active features -- processed seqs: 1233 identifying model active features -- processed seqs: 1234 identifying model active features -- processed seqs: 1235 identifying model active features -- processed seqs: 1236 identifying model active features -- processed seqs: 1237 identifying model active features -- processed seqs: 1238 identifying model active features -- processed seqs: 1239 identifying model active features -- processed seqs: 1240 identifying model active features -- processed seqs: 1241 identifying model active features -- processed seqs: 1242 identifying model active features -- processed seqs: 1243 identifying model active features -- processed seqs: 1244 identifying model active features -- processed seqs: 1245 identifying model active features -- processed seqs: 1246 identifying model active features -- processed seqs: 1247 identifying model active features -- processed seqs: 1248 identifying model active features -- processed seqs: 1249 identifying model active features -- processed seqs: 1250 identifying model active features -- processed seqs: 1251 identifying model active features -- processed seqs: 1252 identifying model active features -- processed seqs: 1253 identifying model active features -- processed seqs: 1254 identifying model active features -- processed seqs: 1255 identifying model active features -- processed seqs: 1256 identifying model active features -- processed seqs: 1257 identifying model active features -- processed seqs: 1258 identifying model active features -- processed seqs: 1259 identifying model active features -- processed seqs: 1260 identifying model active features -- processed seqs: 1261 identifying model active features -- processed seqs: 1262 identifying model active features -- processed seqs: 1263 identifying model active features -- processed seqs: 1264 identifying model active features -- processed seqs: 1265 identifying model active features -- processed seqs: 1266 identifying model active features -- processed seqs: 1267 identifying model active features -- processed seqs: 1268 identifying model active features -- processed seqs: 1269 identifying model active features -- processed seqs: 1270 identifying model active features -- processed seqs: 1271 identifying model active features -- processed seqs: 1272 identifying model active features -- processed seqs: 1273 identifying model active features -- processed seqs: 1274 identifying model active features -- processed seqs: 1275 identifying model active features -- processed seqs: 1276 identifying model active features -- processed seqs: 1277 identifying model active features -- processed seqs: 1278 identifying model active features -- processed seqs: 1279 identifying model active features -- processed seqs: 1280 identifying model active features -- processed seqs: 1281 identifying model active features -- processed seqs: 1282 identifying model active features -- processed seqs: 1283 identifying model active features -- processed seqs: 1284 identifying model active features -- processed seqs: 1285 identifying model active features -- processed seqs: 1286 identifying model active features -- processed seqs: 1287 identifying model active features -- processed seqs: 1288 identifying model active features -- processed seqs: 1289 identifying model active features -- processed seqs: 1290 identifying model active features -- processed seqs: 1291 identifying model active features -- processed seqs: 1292 identifying model active features -- processed seqs: 1293 identifying model active features -- processed seqs: 1294 identifying model active features -- processed seqs: 1295 identifying model active features -- processed seqs: 1296 identifying model active features -- processed seqs: 1297 identifying model active features -- processed seqs: 1298 identifying model active features -- processed seqs: 1299 identifying model active features -- processed seqs: 1300 identifying model active features -- processed seqs: 1301 identifying model active features -- processed seqs: 1302 identifying model active features -- processed seqs: 1303 identifying model active features -- processed seqs: 1304 identifying model active features -- processed seqs: 1305 identifying model active features -- processed seqs: 1306 identifying model active features -- processed seqs: 1307 identifying model active features -- processed seqs: 1308 identifying model active features -- processed seqs: 1309 identifying model active features -- processed seqs: 1310 identifying model active features -- processed seqs: 1311 identifying model active features -- processed seqs: 1312 identifying model active features -- processed seqs: 1313 identifying model active features -- processed seqs: 1314 identifying model active features -- processed seqs: 1315 identifying model active features -- processed seqs: 1316 identifying model active features -- processed seqs: 1317 identifying model active features -- processed seqs: 1318 identifying model active features -- processed seqs: 1319 identifying model active features -- processed seqs: 1320 identifying model active features -- processed seqs: 1321 identifying model active features -- processed seqs: 1322 identifying model active features -- processed seqs: 1323 identifying model active features -- processed seqs: 1324 identifying model active features -- processed seqs: 1325 identifying model active features -- processed seqs: 1326 identifying model active features -- processed seqs: 1327 identifying model active features -- processed seqs: 1328 identifying model active features -- processed seqs: 1329 identifying model active features -- processed seqs: 1330 identifying model active features -- processed seqs: 1331 identifying model active features -- processed seqs: 1332 identifying model active features -- processed seqs: 1333 identifying model active features -- processed seqs: 1334 identifying model active features -- processed seqs: 1335 identifying model active features -- processed seqs: 1336 identifying model active features -- processed seqs: 1337 identifying model active features -- processed seqs: 1338 identifying model active features -- processed seqs: 1339 identifying model active features -- processed seqs: 1340 identifying model active features -- processed seqs: 1341 identifying model active features -- processed seqs: 1342 identifying model active features -- processed seqs: 1343 identifying model active features -- processed seqs: 1344 identifying model active features -- processed seqs: 1345 identifying model active features -- processed seqs: 1346 identifying model active features -- processed seqs: 1347 identifying model active features -- processed seqs: 1348 identifying model active features -- processed seqs: 1349 identifying model active features -- processed seqs: 1350 identifying model active features -- processed seqs: 1351 identifying model active features -- processed seqs: 1352 identifying model active features -- processed seqs: 1353 identifying model active features -- processed seqs: 1354 identifying model active features -- processed seqs: 1355 identifying model active features -- processed seqs: 1356 identifying model active features -- processed seqs: 1357 identifying model active features -- processed seqs: 1358 identifying model active features -- processed seqs: 1359 identifying model active features -- processed seqs: 1360 identifying model active features -- processed seqs: 1361 identifying model active features -- processed seqs: 1362 identifying model active features -- processed seqs: 1363 identifying model active features -- processed seqs: 1364 identifying model active features -- processed seqs: 1365 identifying model active features -- processed seqs: 1366 identifying model active features -- processed seqs: 1367 identifying model active features -- processed seqs: 1368 identifying model active features -- processed seqs: 1369 identifying model active features -- processed seqs: 1370 identifying model active features -- processed seqs: 1371 identifying model active features -- processed seqs: 1372 identifying model active features -- processed seqs: 1373 identifying model active features -- processed seqs: 1374 identifying model active features -- processed seqs: 1375 identifying model active features -- processed seqs: 1376 identifying model active features -- processed seqs: 1377 identifying model active features -- processed seqs: 1378 identifying model active features -- processed seqs: 1379 identifying model active features -- processed seqs: 1380 identifying model active features -- processed seqs: 1381 identifying model active features -- processed seqs: 1382 identifying model active features -- processed seqs: 1383 identifying model active features -- processed seqs: 1384 identifying model active features -- processed seqs: 1385 identifying model active features -- processed seqs: 1386 identifying model active features -- processed seqs: 1387 identifying model active features -- processed seqs: 1388 identifying model active features -- processed seqs: 1389 identifying model active features -- processed seqs: 1390 identifying model active features -- processed seqs: 1391 identifying model active features -- processed seqs: 1392 identifying model active features -- processed seqs: 1393 identifying model active features -- processed seqs: 1394 identifying model active features -- processed seqs: 1395 identifying model active features -- processed seqs: 1396 identifying model active features -- processed seqs: 1397 identifying model active features -- processed seqs: 1398 identifying model active features -- processed seqs: 1399 identifying model active features -- processed seqs: 1400 identifying model active features -- processed seqs: 1401 identifying model active features -- processed seqs: 1402 identifying model active features -- processed seqs: 1403 identifying model active features -- processed seqs: 1404 identifying model active features -- processed seqs: 1405 identifying model active features -- processed seqs: 1406 identifying model active features -- processed seqs: 1407 identifying model active features -- processed seqs: 1408 identifying model active features -- processed seqs: 1409 identifying model active features -- processed seqs: 1410 identifying model active features -- processed seqs: 1411 identifying model active features -- processed seqs: 1412 identifying model active features -- processed seqs: 1413 identifying model active features -- processed seqs: 1414 identifying model active features -- processed seqs: 1415 identifying model active features -- processed seqs: 1416 identifying model active features -- processed seqs: 1417 identifying model active features -- processed seqs: 1418 identifying model active features -- processed seqs: 1419 identifying model active features -- processed seqs: 1420 identifying model active features -- processed seqs: 1421 identifying model active features -- processed seqs: 1422 identifying model active features -- processed seqs: 1423 identifying model active features -- processed seqs: 1424 identifying model active features -- processed seqs: 1425 identifying model active features -- processed seqs: 1426 identifying model active features -- processed seqs: 1427 identifying model active features -- processed seqs: 1428 identifying model active features -- processed seqs: 1429 identifying model active features -- processed seqs: 1430 identifying model active features -- processed seqs: 1431 identifying model active features -- processed seqs: 1432 identifying model active features -- processed seqs: 1433 identifying model active features -- processed seqs: 1434 identifying model active features -- processed seqs: 1435 identifying model active features -- processed seqs: 1436 identifying model active features -- processed seqs: 1437 identifying model active features -- processed seqs: 1438 identifying model active features -- processed seqs: 1439 identifying model active features -- processed seqs: 1440 identifying model active features -- processed seqs: 1441 identifying model active features -- processed seqs: 1442 identifying model active features -- processed seqs: 1443 identifying model active features -- processed seqs: 1444 identifying model active features -- processed seqs: 1445 identifying model active features -- processed seqs: 1446 identifying model active features -- processed seqs: 1447 identifying model active features -- processed seqs: 1448 identifying model active features -- processed seqs: 1449 identifying model active features -- processed seqs: 1450 identifying model active features -- processed seqs: 1451 identifying model active features -- processed seqs: 1452 identifying model active features -- processed seqs: 1453 identifying model active features -- processed seqs: 1454 identifying model active features -- processed seqs: 1455 identifying model active features -- processed seqs: 1456 identifying model active features -- processed seqs: 1457 identifying model active features -- processed seqs: 1458 identifying model active features -- processed seqs: 1459 identifying model active features -- processed seqs: 1460 identifying model active features -- processed seqs: 1461 identifying model active features -- processed seqs: 1462 identifying model active features -- processed seqs: 1463 identifying model active features -- processed seqs: 1464 identifying model active features -- processed seqs: 1465 identifying model active features -- processed seqs: 1466 identifying model active features -- processed seqs: 1467 identifying model active features -- processed seqs: 1468 identifying model active features -- processed seqs: 1469 identifying model active features -- processed seqs: 1470 identifying model active features -- processed seqs: 1471 identifying model active features -- processed seqs: 1472 identifying model active features -- processed seqs: 1473 identifying model active features -- processed seqs: 1474 identifying model active features -- processed seqs: 1475 identifying model active features -- processed seqs: 1476 identifying model active features -- processed seqs: 1477 identifying model active features -- processed seqs: 1478 identifying model active features -- processed seqs: 1479 identifying model active features -- processed seqs: 1480 identifying model active features -- processed seqs: 1481 identifying model active features -- processed seqs: 1482 identifying model active features -- processed seqs: 1483 identifying model active features -- processed seqs: 1484 identifying model active features -- processed seqs: 1485 identifying model active features -- processed seqs: 1486 identifying model active features -- processed seqs: 1487 identifying model active features -- processed seqs: 1488 identifying model active features -- processed seqs: 1489 identifying model active features -- processed seqs: 1490 identifying model active features -- processed seqs: 1491 identifying model active features -- processed seqs: 1492 identifying model active features -- processed seqs: 1493 identifying model active features -- processed seqs: 1494 identifying model active features -- processed seqs: 1495 identifying model active features -- processed seqs: 1496 identifying model active features -- processed seqs: 1497 identifying model active features -- processed seqs: 1498 identifying model active features -- processed seqs: 1499 identifying model active features -- processed seqs: 1500 identifying model active features -- processed seqs: 1501 identifying model active features -- processed seqs: 1502 identifying model active features -- processed seqs: 1503 identifying model active features -- processed seqs: 1504 identifying model active features -- processed seqs: 1505 identifying model active features -- processed seqs: 1506 identifying model active features -- processed seqs: 1507 identifying model active features -- processed seqs: 1508 identifying model active features -- processed seqs: 1509 identifying model active features -- processed seqs: 1510 identifying model active features -- processed seqs: 1511 identifying model active features -- processed seqs: 1512 identifying model active features -- processed seqs: 1513 identifying model active features -- processed seqs: 1514 identifying model active features -- processed seqs: 1515 identifying model active features -- processed seqs: 1516 identifying model active features -- processed seqs: 1517 identifying model active features -- processed seqs: 1518 identifying model active features -- processed seqs: 1519 identifying model active features -- processed seqs: 1520 identifying model active features -- processed seqs: 1521 identifying model active features -- processed seqs: 1522 identifying model active features -- processed seqs: 1523 identifying model active features -- processed seqs: 1524 identifying model active features -- processed seqs: 1525 identifying model active features -- processed seqs: 1526 identifying model active features -- processed seqs: 1527 identifying model active features -- processed seqs: 1528 identifying model active features -- processed seqs: 1529 identifying model active features -- processed seqs: 1530 identifying model active features -- processed seqs: 1531 identifying model active features -- processed seqs: 1532 identifying model active features -- processed seqs: 1533 identifying model active features -- processed seqs: 1534 identifying model active features -- processed seqs: 1535 identifying model active features -- processed seqs: 1536 identifying model active features -- processed seqs: 1537 identifying model active features -- processed seqs: 1538 identifying model active features -- processed seqs: 1539 identifying model active features -- processed seqs: 1540 identifying model active features -- processed seqs: 1541 identifying model active features -- processed seqs: 1542 identifying model active features -- processed seqs: 1543 identifying model active features -- processed seqs: 1544 identifying model active features -- processed seqs: 1545 identifying model active features -- processed seqs: 1546 identifying model active features -- processed seqs: 1547 identifying model active features -- processed seqs: 1548 identifying model active features -- processed seqs: 1549 identifying model active features -- processed seqs: 1550 identifying model active features -- processed seqs: 1551 identifying model active features -- processed seqs: 1552 identifying model active features -- processed seqs: 1553 identifying model active features -- processed seqs: 1554 identifying model active features -- processed seqs: 1555 identifying model active features -- processed seqs: 1556 identifying model active features -- processed seqs: 1557 identifying model active features -- processed seqs: 1558 identifying model active features -- processed seqs: 1559 identifying model active features -- processed seqs: 1560 identifying model active features -- processed seqs: 1561 identifying model active features -- processed seqs: 1562 identifying model active features -- processed seqs: 1563 identifying model active features -- processed seqs: 1564 identifying model active features -- processed seqs: 1565 identifying model active features -- processed seqs: 1566 identifying model active features -- processed seqs: 1567 identifying model active features -- processed seqs: 1568 identifying model active features -- processed seqs: 1569 identifying model active features -- processed seqs: 1570 identifying model active features -- processed seqs: 1571 identifying model active features -- processed seqs: 1572 identifying model active features -- processed seqs: 1573 identifying model active features -- processed seqs: 1574 identifying model active features -- processed seqs: 1575 identifying model active features -- processed seqs: 1576 identifying model active features -- processed seqs: 1577 identifying model active features -- processed seqs: 1578 identifying model active features -- processed seqs: 1579 identifying model active features -- processed seqs: 1580 identifying model active features -- processed seqs: 1581 identifying model active features -- processed seqs: 1582 identifying model active features -- processed seqs: 1583 identifying model active features -- processed seqs: 1584 identifying model active features -- processed seqs: 1585 identifying model active features -- processed seqs: 1586 identifying model active features -- processed seqs: 1587 identifying model active features -- processed seqs: 1588 identifying model active features -- processed seqs: 1589 identifying model active features -- processed seqs: 1590 identifying model active features -- processed seqs: 1591 identifying model active features -- processed seqs: 1592 identifying model active features -- processed seqs: 1593 identifying model active features -- processed seqs: 1594 identifying model active features -- processed seqs: 1595 identifying model active features -- processed seqs: 1596 identifying model active features -- processed seqs: 1597 identifying model active features -- processed seqs: 1598 identifying model active features -- processed seqs: 1599 identifying model active features -- processed seqs: 1600 identifying model active features -- processed seqs: 1601 identifying model active features -- processed seqs: 1602 identifying model active features -- processed seqs: 1603 identifying model active features -- processed seqs: 1604 identifying model active features -- processed seqs: 1605 identifying model active features -- processed seqs: 1606 identifying model active features -- processed seqs: 1607 identifying model active features -- processed seqs: 1608 identifying model active features -- processed seqs: 1609 identifying model active features -- processed seqs: 1610 identifying model active features -- processed seqs: 1611 identifying model active features -- processed seqs: 1612 identifying model active features -- processed seqs: 1613 identifying model active features -- processed seqs: 1614 identifying model active features -- processed seqs: 1615 identifying model active features -- processed seqs: 1616 identifying model active features -- processed seqs: 1617 identifying model active features -- processed seqs: 1618 identifying model active features -- processed seqs: 1619 identifying model active features -- processed seqs: 1620 identifying model active features -- processed seqs: 1621 identifying model active features -- processed seqs: 1622 identifying model active features -- processed seqs: 1623 identifying model active features -- processed seqs: 1624 identifying model active features -- processed seqs: 1625 identifying model active features -- processed seqs: 1626 identifying model active features -- processed seqs: 1627 identifying model active features -- processed seqs: 1628 identifying model active features -- processed seqs: 1629 identifying model active features -- processed seqs: 1630 identifying model active features -- processed seqs: 1631 identifying model active features -- processed seqs: 1632 identifying model active features -- processed seqs: 1633 identifying model active features -- processed seqs: 1634 identifying model active features -- processed seqs: 1635 identifying model active features -- processed seqs: 1636 identifying model active features -- processed seqs: 1637 identifying model active features -- processed seqs: 1638 identifying model active features -- processed seqs: 1639 identifying model active features -- processed seqs: 1640 identifying model active features -- processed seqs: 1641 identifying model active features -- processed seqs: 1642 identifying model active features -- processed seqs: 1643 identifying model active features -- processed seqs: 1644 identifying model active features -- processed seqs: 1645 identifying model active features -- processed seqs: 1646 identifying model active features -- processed seqs: 1647 identifying model active features -- processed seqs: 1648 identifying model active features -- processed seqs: 1649 identifying model active features -- processed seqs: 1650 identifying model active features -- processed seqs: 1651 identifying model active features -- processed seqs: 1652 identifying model active features -- processed seqs: 1653 identifying model active features -- processed seqs: 1654 identifying model active features -- processed seqs: 1655 identifying model active features -- processed seqs: 1656 identifying model active features -- processed seqs: 1657 identifying model active features -- processed seqs: 1658 identifying model active features -- processed seqs: 1659 identifying model active features -- processed seqs: 1660 identifying model active features -- processed seqs: 1661 identifying model active features -- processed seqs: 1662 identifying model active features -- processed seqs: 1663 identifying model active features -- processed seqs: 1664 identifying model active features -- processed seqs: 1665 identifying model active features -- processed seqs: 1666 identifying model active features -- processed seqs: 1667 identifying model active features -- processed seqs: 1668 identifying model active features -- processed seqs: 1669 identifying model active features -- processed seqs: 1670 identifying model active features -- processed seqs: 1671 identifying model active features -- processed seqs: 1672 identifying model active features -- processed seqs: 1673 identifying model active features -- processed seqs: 1674 identifying model active features -- processed seqs: 1675 identifying model active features -- processed seqs: 1676 identifying model active features -- processed seqs: 1677 identifying model active features -- processed seqs: 1678 identifying model active features -- processed seqs: 1679 identifying model active features -- processed seqs: 1680 identifying model active features -- processed seqs: 1681 identifying model active features -- processed seqs: 1682 identifying model active features -- processed seqs: 1683 identifying model active features -- processed seqs: 1684 identifying model active features -- processed seqs: 1685 identifying model active features -- processed seqs: 1686 identifying model active features -- processed seqs: 1687 identifying model active features -- processed seqs: 1688 identifying model active features -- processed seqs: 1689 identifying model active features -- processed seqs: 1690 identifying model active features -- processed seqs: 1691 identifying model active features -- processed seqs: 1692 identifying model active features -- processed seqs: 1693 identifying model active features -- processed seqs: 1694 identifying model active features -- processed seqs: 1695 identifying model active features -- processed seqs: 1696 identifying model active features -- processed seqs: 1697 identifying model active features -- processed seqs: 1698 identifying model active features -- processed seqs: 1699 identifying model active features -- processed seqs: 1700 identifying model active features -- processed seqs: 1701 identifying model active features -- processed seqs: 1702 identifying model active features -- processed seqs: 1703 identifying model active features -- processed seqs: 1704 identifying model active features -- processed seqs: 1705 identifying model active features -- processed seqs: 1706 identifying model active features -- processed seqs: 1707 identifying model active features -- processed seqs: 1708 identifying model active features -- processed seqs: 1709 identifying model active features -- processed seqs: 1710 identifying model active features -- processed seqs: 1711 identifying model active features -- processed seqs: 1712 identifying model active features -- processed seqs: 1713 identifying model active features -- processed seqs: 1714 identifying model active features -- processed seqs: 1715 identifying model active features -- processed seqs: 1716 identifying model active features -- processed seqs: 1717 identifying model active features -- processed seqs: 1718 identifying model active features -- processed seqs: 1719 identifying model active features -- processed seqs: 1720 identifying model active features -- processed seqs: 1721 identifying model active features -- processed seqs: 1722 identifying model active features -- processed seqs: 1723 identifying model active features -- processed seqs: 1724 identifying model active features -- processed seqs: 1725 identifying model active features -- processed seqs: 1726 identifying model active features -- processed seqs: 1727 identifying model active features -- processed seqs: 1728 identifying model active features -- processed seqs: 1729 identifying model active features -- processed seqs: 1730 identifying model active features -- processed seqs: 1731 identifying model active features -- processed seqs: 1732 identifying model active features -- processed seqs: 1733 identifying model active features -- processed seqs: 1734 identifying model active features -- processed seqs: 1735 identifying model active features -- processed seqs: 1736 identifying model active features -- processed seqs: 1737 identifying model active features -- processed seqs: 1738 identifying model active features -- processed seqs: 1739 identifying model active features -- processed seqs: 1740 identifying model active features -- processed seqs: 1741 identifying model active features -- processed seqs: 1742 identifying model active features -- processed seqs: 1743 identifying model active features -- processed seqs: 1744 identifying model active features -- processed seqs: 1745 identifying model active features -- processed seqs: 1746 identifying model active features -- processed seqs: 1747 identifying model active features -- processed seqs: 1748 identifying model active features -- processed seqs: 1749 identifying model active features -- processed seqs: 1750 identifying model active features -- processed seqs: 1751 identifying model active features -- processed seqs: 1752 identifying model active features -- processed seqs: 1753 identifying model active features -- processed seqs: 1754 identifying model active features -- processed seqs: 1755 identifying model active features -- processed seqs: 1756 identifying model active features -- processed seqs: 1757 identifying model active features -- processed seqs: 1758 identifying model active features -- processed seqs: 1759 identifying model active features -- processed seqs: 1760 identifying model active features -- processed seqs: 1761 identifying model active features -- processed seqs: 1762 identifying model active features -- processed seqs: 1763 identifying model active features -- processed seqs: 1764 identifying model active features -- processed seqs: 1765 identifying model active features -- processed seqs: 1766 identifying model active features -- processed seqs: 1767 identifying model active features -- processed seqs: 1768 identifying model active features -- processed seqs: 1769 identifying model active features -- processed seqs: 1770 identifying model active features -- processed seqs: 1771 identifying model active features -- processed seqs: 1772 identifying model active features -- processed seqs: 1773 identifying model active features -- processed seqs: 1774 identifying model active features -- processed seqs: 1775 identifying model active features -- processed seqs: 1776 identifying model active features -- processed seqs: 1777 identifying model active features -- processed seqs: 1778 identifying model active features -- processed seqs: 1779 identifying model active features -- processed seqs: 1780 identifying model active features -- processed seqs: 1781 identifying model active features -- processed seqs: 1782 identifying model active features -- processed seqs: 1783 identifying model active features -- processed seqs: 1784 identifying model active features -- processed seqs: 1785 identifying model active features -- processed seqs: 1786 identifying model active features -- processed seqs: 1787 identifying model active features -- processed seqs: 1788 identifying model active features -- processed seqs: 1789 identifying model active features -- processed seqs: 1790 identifying model active features -- processed seqs: 1791 identifying model active features -- processed seqs: 1792 identifying model active features -- processed seqs: 1793 identifying model active features -- processed seqs: 1794 identifying model active features -- processed seqs: 1795 identifying model active features -- processed seqs: 1796 identifying model active features -- processed seqs: 1797 identifying model active features -- processed seqs: 1798 identifying model active features -- processed seqs: 1799 identifying model active features -- processed seqs: 1800 identifying model active features -- processed seqs: 1801 identifying model active features -- processed seqs: 1802 identifying model active features -- processed seqs: 1803 identifying model active features -- processed seqs: 1804 identifying model active features -- processed seqs: 1805 identifying model active features -- processed seqs: 1806 identifying model active features -- processed seqs: 1807 identifying model active features -- processed seqs: 1808 identifying model active features -- processed seqs: 1809 identifying model active features -- processed seqs: 1810 identifying model active features -- processed seqs: 1811 identifying model active features -- processed seqs: 1812 identifying model active features -- processed seqs: 1813 identifying model active features -- processed seqs: 1814 identifying model active features -- processed seqs: 1815 identifying model active features -- processed seqs: 1816 identifying model active features -- processed seqs: 1817 identifying model active features -- processed seqs: 1818 identifying model active features -- processed seqs: 1819 identifying model active features -- processed seqs: 1820 identifying model active features -- processed seqs: 1821 identifying model active features -- processed seqs: 1822 identifying model active features -- processed seqs: 1823 identifying model active features -- processed seqs: 1824 identifying model active features -- processed seqs: 1825 identifying model active features -- processed seqs: 1826 identifying model active features -- processed seqs: 1827 identifying model active features -- processed seqs: 1828 identifying model active features -- processed seqs: 1829 identifying model active features -- processed seqs: 1830 identifying model active features -- processed seqs: 1831 identifying model active features -- processed seqs: 1832 identifying model active features -- processed seqs: 1833 identifying model active features -- processed seqs: 1834 identifying model active features -- processed seqs: 1835 identifying model active features -- processed seqs: 1836 identifying model active features -- processed seqs: 1837 identifying model active features -- processed seqs: 1838 identifying model active features -- processed seqs: 1839 sequence decoded -- 1838 sequences are left sequence decoded -- 1837 sequences are left sequence decoded -- 1836 sequences are left sequence decoded -- 1835 sequences are left sequence decoded -- 1834 sequences are left sequence decoded -- 1833 sequences are left sequence decoded -- 1832 sequences are left sequence decoded -- 1831 sequences are left sequence decoded -- 1830 sequences are left sequence decoded -- 1829 sequences are left sequence decoded -- 1828 sequences are left sequence decoded -- 1827 sequences are left sequence decoded -- 1826 sequences are left sequence decoded -- 1825 sequences are left sequence decoded -- 1824 sequences are left sequence decoded -- 1823 sequences are left sequence decoded -- 1822 sequences are left sequence decoded -- 1821 sequences are left sequence decoded -- 1820 sequences are left sequence decoded -- 1819 sequences are left sequence decoded -- 1818 sequences are left sequence decoded -- 1817 sequences are left sequence decoded -- 1816 sequences are left sequence decoded -- 1815 sequences are left sequence decoded -- 1814 sequences are left sequence decoded -- 1813 sequences are left sequence decoded -- 1812 sequences are left sequence decoded -- 1811 sequences are left sequence decoded -- 1810 sequences are left sequence decoded -- 1809 sequences are left sequence decoded -- 1808 sequences are left sequence decoded -- 1807 sequences are left sequence decoded -- 1806 sequences are left sequence decoded -- 1805 sequences are left sequence decoded -- 1804 sequences are left sequence decoded -- 1803 sequences are left sequence decoded -- 1802 sequences are left sequence decoded -- 1801 sequences are left sequence decoded -- 1800 sequences are left sequence decoded -- 1799 sequences are left sequence decoded -- 1798 sequences are left sequence decoded -- 1797 sequences are left sequence decoded -- 1796 sequences are left sequence decoded -- 1795 sequences are left sequence decoded -- 1794 sequences are left sequence decoded -- 1793 sequences are left sequence decoded -- 1792 sequences are left sequence decoded -- 1791 sequences are left sequence decoded -- 1790 sequences are left sequence decoded -- 1789 sequences are left sequence decoded -- 1788 sequences are left sequence decoded -- 1787 sequences are left sequence decoded -- 1786 sequences are left sequence decoded -- 1785 sequences are left sequence decoded -- 1784 sequences are left sequence decoded -- 1783 sequences are left sequence decoded -- 1782 sequences are left sequence decoded -- 1781 sequences are left sequence decoded -- 1780 sequences are left sequence decoded -- 1779 sequences are left sequence decoded -- 1778 sequences are left sequence decoded -- 1777 sequences are left sequence decoded -- 1776 sequences are left sequence decoded -- 1775 sequences are left sequence decoded -- 1774 sequences are left sequence decoded -- 1773 sequences are left sequence decoded -- 1772 sequences are left sequence decoded -- 1771 sequences are left sequence decoded -- 1770 sequences are left sequence decoded -- 1769 sequences are left sequence decoded -- 1768 sequences are left sequence decoded -- 1767 sequences are left sequence decoded -- 1766 sequences are left sequence decoded -- 1765 sequences are left sequence decoded -- 1764 sequences are left sequence decoded -- 1763 sequences are left sequence decoded -- 1762 sequences are left sequence decoded -- 1761 sequences are left sequence decoded -- 1760 sequences are left sequence decoded -- 1759 sequences are left sequence decoded -- 1758 sequences are left sequence decoded -- 1757 sequences are left sequence decoded -- 1756 sequences are left sequence decoded -- 1755 sequences are left sequence decoded -- 1754 sequences are left sequence decoded -- 1753 sequences are left sequence decoded -- 1752 sequences are left sequence decoded -- 1751 sequences are left sequence decoded -- 1750 sequences are left sequence decoded -- 1749 sequences are left sequence decoded -- 1748 sequences are left sequence decoded -- 1747 sequences are left sequence decoded -- 1746 sequences are left sequence decoded -- 1745 sequences are left sequence decoded -- 1744 sequences are left sequence decoded -- 1743 sequences are left sequence decoded -- 1742 sequences are left sequence decoded -- 1741 sequences are left sequence decoded -- 1740 sequences are left sequence decoded -- 1739 sequences are left sequence decoded -- 1738 sequences are left sequence decoded -- 1737 sequences are left sequence decoded -- 1736 sequences are left sequence decoded -- 1735 sequences are left sequence decoded -- 1734 sequences are left sequence decoded -- 1733 sequences are left sequence decoded -- 1732 sequences are left sequence decoded -- 1731 sequences are left sequence decoded -- 1730 sequences are left sequence decoded -- 1729 sequences are left sequence decoded -- 1728 sequences are left sequence decoded -- 1727 sequences are left sequence decoded -- 1726 sequences are left sequence decoded -- 1725 sequences are left sequence decoded -- 1724 sequences are left sequence decoded -- 1723 sequences are left sequence decoded -- 1722 sequences are left sequence decoded -- 1721 sequences are left sequence decoded -- 1720 sequences are left sequence decoded -- 1719 sequences are left sequence decoded -- 1718 sequences are left sequence decoded -- 1717 sequences are left sequence decoded -- 1716 sequences are left sequence decoded -- 1715 sequences are left sequence decoded -- 1714 sequences are left sequence decoded -- 1713 sequences are left sequence decoded -- 1712 sequences are left sequence decoded -- 1711 sequences are left sequence decoded -- 1710 sequences are left sequence decoded -- 1709 sequences are left sequence decoded -- 1708 sequences are left sequence decoded -- 1707 sequences are left sequence decoded -- 1706 sequences are left sequence decoded -- 1705 sequences are left sequence decoded -- 1704 sequences are left sequence decoded -- 1703 sequences are left sequence decoded -- 1702 sequences are left sequence decoded -- 1701 sequences are left sequence decoded -- 1700 sequences are left sequence decoded -- 1699 sequences are left sequence decoded -- 1698 sequences are left sequence decoded -- 1697 sequences are left sequence decoded -- 1696 sequences are left sequence decoded -- 1695 sequences are left sequence decoded -- 1694 sequences are left sequence decoded -- 1693 sequences are left sequence decoded -- 1692 sequences are left sequence decoded -- 1691 sequences are left sequence decoded -- 1690 sequences are left sequence decoded -- 1689 sequences are left sequence decoded -- 1688 sequences are left sequence decoded -- 1687 sequences are left sequence decoded -- 1686 sequences are left sequence decoded -- 1685 sequences are left sequence decoded -- 1684 sequences are left sequence decoded -- 1683 sequences are left sequence decoded -- 1682 sequences are left sequence decoded -- 1681 sequences are left sequence decoded -- 1680 sequences are left sequence decoded -- 1679 sequences are left sequence decoded -- 1678 sequences are left sequence decoded -- 1677 sequences are left sequence decoded -- 1676 sequences are left sequence decoded -- 1675 sequences are left sequence decoded -- 1674 sequences are left sequence decoded -- 1673 sequences are left sequence decoded -- 1672 sequences are left sequence decoded -- 1671 sequences are left sequence decoded -- 1670 sequences are left sequence decoded -- 1669 sequences are left sequence decoded -- 1668 sequences are left sequence decoded -- 1667 sequences are left sequence decoded -- 1666 sequences are left sequence decoded -- 1665 sequences are left sequence decoded -- 1664 sequences are left sequence decoded -- 1663 sequences are left sequence decoded -- 1662 sequences are left sequence decoded -- 1661 sequences are left sequence decoded -- 1660 sequences are left sequence decoded -- 1659 sequences are left sequence decoded -- 1658 sequences are left sequence decoded -- 1657 sequences are left sequence decoded -- 1656 sequences are left sequence decoded -- 1655 sequences are left sequence decoded -- 1654 sequences are left sequence decoded -- 1653 sequences are left sequence decoded -- 1652 sequences are left sequence decoded -- 1651 sequences are left sequence decoded -- 1650 sequences are left sequence decoded -- 1649 sequences are left sequence decoded -- 1648 sequences are left sequence decoded -- 1647 sequences are left sequence decoded -- 1646 sequences are left sequence decoded -- 1645 sequences are left sequence decoded -- 1644 sequences are left sequence decoded -- 1643 sequences are left sequence decoded -- 1642 sequences are left sequence decoded -- 1641 sequences are left sequence decoded -- 1640 sequences are left sequence decoded -- 1639 sequences are left sequence decoded -- 1638 sequences are left sequence decoded -- 1637 sequences are left sequence decoded -- 1636 sequences are left sequence decoded -- 1635 sequences are left sequence decoded -- 1634 sequences are left sequence decoded -- 1633 sequences are left sequence decoded -- 1632 sequences are left sequence decoded -- 1631 sequences are left sequence decoded -- 1630 sequences are left sequence decoded -- 1629 sequences are left sequence decoded -- 1628 sequences are left sequence decoded -- 1627 sequences are left sequence decoded -- 1626 sequences are left sequence decoded -- 1625 sequences are left sequence decoded -- 1624 sequences are left sequence decoded -- 1623 sequences are left sequence decoded -- 1622 sequences are left sequence decoded -- 1621 sequences are left sequence decoded -- 1620 sequences are left sequence decoded -- 1619 sequences are left sequence decoded -- 1618 sequences are left sequence decoded -- 1617 sequences are left sequence decoded -- 1616 sequences are left sequence decoded -- 1615 sequences are left sequence decoded -- 1614 sequences are left sequence decoded -- 1613 sequences are left sequence decoded -- 1612 sequences are left sequence decoded -- 1611 sequences are left sequence decoded -- 1610 sequences are left sequence decoded -- 1609 sequences are left sequence decoded -- 1608 sequences are left sequence decoded -- 1607 sequences are left sequence decoded -- 1606 sequences are left sequence decoded -- 1605 sequences are left sequence decoded -- 1604 sequences are left sequence decoded -- 1603 sequences are left sequence decoded -- 1602 sequences are left sequence decoded -- 1601 sequences are left sequence decoded -- 1600 sequences are left sequence decoded -- 1599 sequences are left sequence decoded -- 1598 sequences are left sequence decoded -- 1597 sequences are left sequence decoded -- 1596 sequences are left sequence decoded -- 1595 sequences are left sequence decoded -- 1594 sequences are left sequence decoded -- 1593 sequences are left sequence decoded -- 1592 sequences are left sequence decoded -- 1591 sequences are left sequence decoded -- 1590 sequences are left sequence decoded -- 1589 sequences are left sequence decoded -- 1588 sequences are left sequence decoded -- 1587 sequences are left sequence decoded -- 1586 sequences are left sequence decoded -- 1585 sequences are left sequence decoded -- 1584 sequences are left sequence decoded -- 1583 sequences are left sequence decoded -- 1582 sequences are left sequence decoded -- 1581 sequences are left sequence decoded -- 1580 sequences are left sequence decoded -- 1579 sequences are left sequence decoded -- 1578 sequences are left sequence decoded -- 1577 sequences are left sequence decoded -- 1576 sequences are left sequence decoded -- 1575 sequences are left sequence decoded -- 1574 sequences are left sequence decoded -- 1573 sequences are left sequence decoded -- 1572 sequences are left sequence decoded -- 1571 sequences are left sequence decoded -- 1570 sequences are left sequence decoded -- 1569 sequences are left sequence decoded -- 1568 sequences are left sequence decoded -- 1567 sequences are left sequence decoded -- 1566 sequences are left sequence decoded -- 1565 sequences are left sequence decoded -- 1564 sequences are left sequence decoded -- 1563 sequences are left sequence decoded -- 1562 sequences are left sequence decoded -- 1561 sequences are left sequence decoded -- 1560 sequences are left sequence decoded -- 1559 sequences are left sequence decoded -- 1558 sequences are left sequence decoded -- 1557 sequences are left sequence decoded -- 1556 sequences are left sequence decoded -- 1555 sequences are left sequence decoded -- 1554 sequences are left sequence decoded -- 1553 sequences are left sequence decoded -- 1552 sequences are left sequence decoded -- 1551 sequences are left sequence decoded -- 1550 sequences are left sequence decoded -- 1549 sequences are left sequence decoded -- 1548 sequences are left sequence decoded -- 1547 sequences are left sequence decoded -- 1546 sequences are left sequence decoded -- 1545 sequences are left sequence decoded -- 1544 sequences are left sequence decoded -- 1543 sequences are left sequence decoded -- 1542 sequences are left sequence decoded -- 1541 sequences are left sequence decoded -- 1540 sequences are left sequence decoded -- 1539 sequences are left sequence decoded -- 1538 sequences are left sequence decoded -- 1537 sequences are left sequence decoded -- 1536 sequences are left sequence decoded -- 1535 sequences are left sequence decoded -- 1534 sequences are left sequence decoded -- 1533 sequences are left sequence decoded -- 1532 sequences are left sequence decoded -- 1531 sequences are left sequence decoded -- 1530 sequences are left sequence decoded -- 1529 sequences are left sequence decoded -- 1528 sequences are left sequence decoded -- 1527 sequences are left sequence decoded -- 1526 sequences are left sequence decoded -- 1525 sequences are left sequence decoded -- 1524 sequences are left sequence decoded -- 1523 sequences are left sequence decoded -- 1522 sequences are left sequence decoded -- 1521 sequences are left sequence decoded -- 1520 sequences are left sequence decoded -- 1519 sequences are left sequence decoded -- 1518 sequences are left sequence decoded -- 1517 sequences are left sequence decoded -- 1516 sequences are left sequence decoded -- 1515 sequences are left sequence decoded -- 1514 sequences are left sequence decoded -- 1513 sequences are left sequence decoded -- 1512 sequences are left sequence decoded -- 1511 sequences are left sequence decoded -- 1510 sequences are left sequence decoded -- 1509 sequences are left sequence decoded -- 1508 sequences are left sequence decoded -- 1507 sequences are left sequence decoded -- 1506 sequences are left sequence decoded -- 1505 sequences are left sequence decoded -- 1504 sequences are left sequence decoded -- 1503 sequences are left sequence decoded -- 1502 sequences are left sequence decoded -- 1501 sequences are left sequence decoded -- 1500 sequences are left sequence decoded -- 1499 sequences are left sequence decoded -- 1498 sequences are left sequence decoded -- 1497 sequences are left sequence decoded -- 1496 sequences are left sequence decoded -- 1495 sequences are left sequence decoded -- 1494 sequences are left sequence decoded -- 1493 sequences are left sequence decoded -- 1492 sequences are left sequence decoded -- 1491 sequences are left sequence decoded -- 1490 sequences are left sequence decoded -- 1489 sequences are left sequence decoded -- 1488 sequences are left sequence decoded -- 1487 sequences are left sequence decoded -- 1486 sequences are left sequence decoded -- 1485 sequences are left sequence decoded -- 1484 sequences are left sequence decoded -- 1483 sequences are left sequence decoded -- 1482 sequences are left sequence decoded -- 1481 sequences are left sequence decoded -- 1480 sequences are left sequence decoded -- 1479 sequences are left sequence decoded -- 1478 sequences are left sequence decoded -- 1477 sequences are left sequence decoded -- 1476 sequences are left sequence decoded -- 1475 sequences are left sequence decoded -- 1474 sequences are left sequence decoded -- 1473 sequences are left sequence decoded -- 1472 sequences are left sequence decoded -- 1471 sequences are left sequence decoded -- 1470 sequences are left sequence decoded -- 1469 sequences are left sequence decoded -- 1468 sequences are left sequence decoded -- 1467 sequences are left sequence decoded -- 1466 sequences are left sequence decoded -- 1465 sequences are left sequence decoded -- 1464 sequences are left sequence decoded -- 1463 sequences are left sequence decoded -- 1462 sequences are left sequence decoded -- 1461 sequences are left sequence decoded -- 1460 sequences are left sequence decoded -- 1459 sequences are left sequence decoded -- 1458 sequences are left sequence decoded -- 1457 sequences are left sequence decoded -- 1456 sequences are left sequence decoded -- 1455 sequences are left sequence decoded -- 1454 sequences are left sequence decoded -- 1453 sequences are left sequence decoded -- 1452 sequences are left sequence decoded -- 1451 sequences are left sequence decoded -- 1450 sequences are left sequence decoded -- 1449 sequences are left sequence decoded -- 1448 sequences are left sequence decoded -- 1447 sequences are left sequence decoded -- 1446 sequences are left sequence decoded -- 1445 sequences are left sequence decoded -- 1444 sequences are left sequence decoded -- 1443 sequences are left sequence decoded -- 1442 sequences are left sequence decoded -- 1441 sequences are left sequence decoded -- 1440 sequences are left sequence decoded -- 1439 sequences are left sequence decoded -- 1438 sequences are left sequence decoded -- 1437 sequences are left sequence decoded -- 1436 sequences are left sequence decoded -- 1435 sequences are left sequence decoded -- 1434 sequences are left sequence decoded -- 1433 sequences are left sequence decoded -- 1432 sequences are left sequence decoded -- 1431 sequences are left sequence decoded -- 1430 sequences are left sequence decoded -- 1429 sequences are left sequence decoded -- 1428 sequences are left sequence decoded -- 1427 sequences are left sequence decoded -- 1426 sequences are left sequence decoded -- 1425 sequences are left sequence decoded -- 1424 sequences are left sequence decoded -- 1423 sequences are left sequence decoded -- 1422 sequences are left sequence decoded -- 1421 sequences are left sequence decoded -- 1420 sequences are left sequence decoded -- 1419 sequences are left sequence decoded -- 1418 sequences are left sequence decoded -- 1417 sequences are left sequence decoded -- 1416 sequences are left sequence decoded -- 1415 sequences are left sequence decoded -- 1414 sequences are left sequence decoded -- 1413 sequences are left sequence decoded -- 1412 sequences are left sequence decoded -- 1411 sequences are left sequence decoded -- 1410 sequences are left sequence decoded -- 1409 sequences are left sequence decoded -- 1408 sequences are left sequence decoded -- 1407 sequences are left sequence decoded -- 1406 sequences are left sequence decoded -- 1405 sequences are left sequence decoded -- 1404 sequences are left sequence decoded -- 1403 sequences are left sequence decoded -- 1402 sequences are left sequence decoded -- 1401 sequences are left sequence decoded -- 1400 sequences are left sequence decoded -- 1399 sequences are left sequence decoded -- 1398 sequences are left sequence decoded -- 1397 sequences are left sequence decoded -- 1396 sequences are left sequence decoded -- 1395 sequences are left sequence decoded -- 1394 sequences are left sequence decoded -- 1393 sequences are left sequence decoded -- 1392 sequences are left sequence decoded -- 1391 sequences are left sequence decoded -- 1390 sequences are left sequence decoded -- 1389 sequences are left sequence decoded -- 1388 sequences are left sequence decoded -- 1387 sequences are left sequence decoded -- 1386 sequences are left sequence decoded -- 1385 sequences are left sequence decoded -- 1384 sequences are left sequence decoded -- 1383 sequences are left sequence decoded -- 1382 sequences are left sequence decoded -- 1381 sequences are left sequence decoded -- 1380 sequences are left sequence decoded -- 1379 sequences are left sequence decoded -- 1378 sequences are left sequence decoded -- 1377 sequences are left sequence decoded -- 1376 sequences are left sequence decoded -- 1375 sequences are left sequence decoded -- 1374 sequences are left sequence decoded -- 1373 sequences are left sequence decoded -- 1372 sequences are left sequence decoded -- 1371 sequences are left sequence decoded -- 1370 sequences are left sequence decoded -- 1369 sequences are left sequence decoded -- 1368 sequences are left sequence decoded -- 1367 sequences are left sequence decoded -- 1366 sequences are left sequence decoded -- 1365 sequences are left sequence decoded -- 1364 sequences are left sequence decoded -- 1363 sequences are left sequence decoded -- 1362 sequences are left sequence decoded -- 1361 sequences are left sequence decoded -- 1360 sequences are left sequence decoded -- 1359 sequences are left sequence decoded -- 1358 sequences are left sequence decoded -- 1357 sequences are left sequence decoded -- 1356 sequences are left sequence decoded -- 1355 sequences are left sequence decoded -- 1354 sequences are left sequence decoded -- 1353 sequences are left sequence decoded -- 1352 sequences are left sequence decoded -- 1351 sequences are left sequence decoded -- 1350 sequences are left sequence decoded -- 1349 sequences are left sequence decoded -- 1348 sequences are left sequence decoded -- 1347 sequences are left sequence decoded -- 1346 sequences are left sequence decoded -- 1345 sequences are left sequence decoded -- 1344 sequences are left sequence decoded -- 1343 sequences are left sequence decoded -- 1342 sequences are left sequence decoded -- 1341 sequences are left sequence decoded -- 1340 sequences are left sequence decoded -- 1339 sequences are left sequence decoded -- 1338 sequences are left sequence decoded -- 1337 sequences are left sequence decoded -- 1336 sequences are left sequence decoded -- 1335 sequences are left sequence decoded -- 1334 sequences are left sequence decoded -- 1333 sequences are left sequence decoded -- 1332 sequences are left sequence decoded -- 1331 sequences are left sequence decoded -- 1330 sequences are left sequence decoded -- 1329 sequences are left sequence decoded -- 1328 sequences are left sequence decoded -- 1327 sequences are left sequence decoded -- 1326 sequences are left sequence decoded -- 1325 sequences are left sequence decoded -- 1324 sequences are left sequence decoded -- 1323 sequences are left sequence decoded -- 1322 sequences are left sequence decoded -- 1321 sequences are left sequence decoded -- 1320 sequences are left sequence decoded -- 1319 sequences are left sequence decoded -- 1318 sequences are left sequence decoded -- 1317 sequences are left sequence decoded -- 1316 sequences are left sequence decoded -- 1315 sequences are left sequence decoded -- 1314 sequences are left sequence decoded -- 1313 sequences are left sequence decoded -- 1312 sequences are left sequence decoded -- 1311 sequences are left sequence decoded -- 1310 sequences are left sequence decoded -- 1309 sequences are left sequence decoded -- 1308 sequences are left sequence decoded -- 1307 sequences are left sequence decoded -- 1306 sequences are left sequence decoded -- 1305 sequences are left sequence decoded -- 1304 sequences are left sequence decoded -- 1303 sequences are left sequence decoded -- 1302 sequences are left sequence decoded -- 1301 sequences are left sequence decoded -- 1300 sequences are left sequence decoded -- 1299 sequences are left sequence decoded -- 1298 sequences are left sequence decoded -- 1297 sequences are left sequence decoded -- 1296 sequences are left sequence decoded -- 1295 sequences are left sequence decoded -- 1294 sequences are left sequence decoded -- 1293 sequences are left sequence decoded -- 1292 sequences are left sequence decoded -- 1291 sequences are left sequence decoded -- 1290 sequences are left sequence decoded -- 1289 sequences are left sequence decoded -- 1288 sequences are left sequence decoded -- 1287 sequences are left sequence decoded -- 1286 sequences are left sequence decoded -- 1285 sequences are left sequence decoded -- 1284 sequences are left sequence decoded -- 1283 sequences are left sequence decoded -- 1282 sequences are left sequence decoded -- 1281 sequences are left sequence decoded -- 1280 sequences are left sequence decoded -- 1279 sequences are left sequence decoded -- 1278 sequences are left sequence decoded -- 1277 sequences are left sequence decoded -- 1276 sequences are left sequence decoded -- 1275 sequences are left sequence decoded -- 1274 sequences are left sequence decoded -- 1273 sequences are left sequence decoded -- 1272 sequences are left sequence decoded -- 1271 sequences are left sequence decoded -- 1270 sequences are left sequence decoded -- 1269 sequences are left sequence decoded -- 1268 sequences are left sequence decoded -- 1267 sequences are left sequence decoded -- 1266 sequences are left sequence decoded -- 1265 sequences are left sequence decoded -- 1264 sequences are left sequence decoded -- 1263 sequences are left sequence decoded -- 1262 sequences are left sequence decoded -- 1261 sequences are left sequence decoded -- 1260 sequences are left sequence decoded -- 1259 sequences are left sequence decoded -- 1258 sequences are left sequence decoded -- 1257 sequences are left sequence decoded -- 1256 sequences are left sequence decoded -- 1255 sequences are left sequence decoded -- 1254 sequences are left sequence decoded -- 1253 sequences are left sequence decoded -- 1252 sequences are left sequence decoded -- 1251 sequences are left sequence decoded -- 1250 sequences are left sequence decoded -- 1249 sequences are left sequence decoded -- 1248 sequences are left sequence decoded -- 1247 sequences are left sequence decoded -- 1246 sequences are left sequence decoded -- 1245 sequences are left sequence decoded -- 1244 sequences are left sequence decoded -- 1243 sequences are left sequence decoded -- 1242 sequences are left sequence decoded -- 1241 sequences are left sequence decoded -- 1240 sequences are left sequence decoded -- 1239 sequences are left sequence decoded -- 1238 sequences are left sequence decoded -- 1237 sequences are left sequence decoded -- 1236 sequences are left sequence decoded -- 1235 sequences are left sequence decoded -- 1234 sequences are left sequence decoded -- 1233 sequences are left sequence decoded -- 1232 sequences are left sequence decoded -- 1231 sequences are left sequence decoded -- 1230 sequences are left sequence decoded -- 1229 sequences are left sequence decoded -- 1228 sequences are left sequence decoded -- 1227 sequences are left sequence decoded -- 1226 sequences are left sequence decoded -- 1225 sequences are left sequence decoded -- 1224 sequences are left sequence decoded -- 1223 sequences are left sequence decoded -- 1222 sequences are left sequence decoded -- 1221 sequences are left sequence decoded -- 1220 sequences are left sequence decoded -- 1219 sequences are left sequence decoded -- 1218 sequences are left sequence decoded -- 1217 sequences are left sequence decoded -- 1216 sequences are left sequence decoded -- 1215 sequences are left sequence decoded -- 1214 sequences are left sequence decoded -- 1213 sequences are left sequence decoded -- 1212 sequences are left sequence decoded -- 1211 sequences are left sequence decoded -- 1210 sequences are left sequence decoded -- 1209 sequences are left sequence decoded -- 1208 sequences are left sequence decoded -- 1207 sequences are left sequence decoded -- 1206 sequences are left sequence decoded -- 1205 sequences are left sequence decoded -- 1204 sequences are left sequence decoded -- 1203 sequences are left sequence decoded -- 1202 sequences are left sequence decoded -- 1201 sequences are left sequence decoded -- 1200 sequences are left sequence decoded -- 1199 sequences are left sequence decoded -- 1198 sequences are left sequence decoded -- 1197 sequences are left sequence decoded -- 1196 sequences are left sequence decoded -- 1195 sequences are left sequence decoded -- 1194 sequences are left sequence decoded -- 1193 sequences are left sequence decoded -- 1192 sequences are left sequence decoded -- 1191 sequences are left sequence decoded -- 1190 sequences are left sequence decoded -- 1189 sequences are left sequence decoded -- 1188 sequences are left sequence decoded -- 1187 sequences are left sequence decoded -- 1186 sequences are left sequence decoded -- 1185 sequences are left sequence decoded -- 1184 sequences are left sequence decoded -- 1183 sequences are left sequence decoded -- 1182 sequences are left sequence decoded -- 1181 sequences are left sequence decoded -- 1180 sequences are left sequence decoded -- 1179 sequences are left sequence decoded -- 1178 sequences are left sequence decoded -- 1177 sequences are left sequence decoded -- 1176 sequences are left sequence decoded -- 1175 sequences are left sequence decoded -- 1174 sequences are left sequence decoded -- 1173 sequences are left sequence decoded -- 1172 sequences are left sequence decoded -- 1171 sequences are left sequence decoded -- 1170 sequences are left sequence decoded -- 1169 sequences are left sequence decoded -- 1168 sequences are left sequence decoded -- 1167 sequences are left sequence decoded -- 1166 sequences are left sequence decoded -- 1165 sequences are left sequence decoded -- 1164 sequences are left sequence decoded -- 1163 sequences are left sequence decoded -- 1162 sequences are left sequence decoded -- 1161 sequences are left sequence decoded -- 1160 sequences are left sequence decoded -- 1159 sequences are left sequence decoded -- 1158 sequences are left sequence decoded -- 1157 sequences are left sequence decoded -- 1156 sequences are left sequence decoded -- 1155 sequences are left sequence decoded -- 1154 sequences are left sequence decoded -- 1153 sequences are left sequence decoded -- 1152 sequences are left sequence decoded -- 1151 sequences are left sequence decoded -- 1150 sequences are left sequence decoded -- 1149 sequences are left sequence decoded -- 1148 sequences are left sequence decoded -- 1147 sequences are left sequence decoded -- 1146 sequences are left sequence decoded -- 1145 sequences are left sequence decoded -- 1144 sequences are left sequence decoded -- 1143 sequences are left sequence decoded -- 1142 sequences are left sequence decoded -- 1141 sequences are left sequence decoded -- 1140 sequences are left sequence decoded -- 1139 sequences are left sequence decoded -- 1138 sequences are left sequence decoded -- 1137 sequences are left sequence decoded -- 1136 sequences are left sequence decoded -- 1135 sequences are left sequence decoded -- 1134 sequences are left sequence decoded -- 1133 sequences are left sequence decoded -- 1132 sequences are left sequence decoded -- 1131 sequences are left sequence decoded -- 1130 sequences are left sequence decoded -- 1129 sequences are left sequence decoded -- 1128 sequences are left sequence decoded -- 1127 sequences are left sequence decoded -- 1126 sequences are left sequence decoded -- 1125 sequences are left sequence decoded -- 1124 sequences are left sequence decoded -- 1123 sequences are left sequence decoded -- 1122 sequences are left sequence decoded -- 1121 sequences are left sequence decoded -- 1120 sequences are left sequence decoded -- 1119 sequences are left sequence decoded -- 1118 sequences are left sequence decoded -- 1117 sequences are left sequence decoded -- 1116 sequences are left sequence decoded -- 1115 sequences are left sequence decoded -- 1114 sequences are left sequence decoded -- 1113 sequences are left sequence decoded -- 1112 sequences are left sequence decoded -- 1111 sequences are left sequence decoded -- 1110 sequences are left sequence decoded -- 1109 sequences are left sequence decoded -- 1108 sequences are left sequence decoded -- 1107 sequences are left sequence decoded -- 1106 sequences are left sequence decoded -- 1105 sequences are left sequence decoded -- 1104 sequences are left sequence decoded -- 1103 sequences are left sequence decoded -- 1102 sequences are left sequence decoded -- 1101 sequences are left sequence decoded -- 1100 sequences are left sequence decoded -- 1099 sequences are left sequence decoded -- 1098 sequences are left sequence decoded -- 1097 sequences are left sequence decoded -- 1096 sequences are left sequence decoded -- 1095 sequences are left sequence decoded -- 1094 sequences are left sequence decoded -- 1093 sequences are left sequence decoded -- 1092 sequences are left sequence decoded -- 1091 sequences are left sequence decoded -- 1090 sequences are left sequence decoded -- 1089 sequences are left sequence decoded -- 1088 sequences are left sequence decoded -- 1087 sequences are left sequence decoded -- 1086 sequences are left sequence decoded -- 1085 sequences are left sequence decoded -- 1084 sequences are left sequence decoded -- 1083 sequences are left sequence decoded -- 1082 sequences are left sequence decoded -- 1081 sequences are left sequence decoded -- 1080 sequences are left sequence decoded -- 1079 sequences are left sequence decoded -- 1078 sequences are left sequence decoded -- 1077 sequences are left sequence decoded -- 1076 sequences are left sequence decoded -- 1075 sequences are left sequence decoded -- 1074 sequences are left sequence decoded -- 1073 sequences are left sequence decoded -- 1072 sequences are left sequence decoded -- 1071 sequences are left sequence decoded -- 1070 sequences are left sequence decoded -- 1069 sequences are left sequence decoded -- 1068 sequences are left sequence decoded -- 1067 sequences are left sequence decoded -- 1066 sequences are left sequence decoded -- 1065 sequences are left sequence decoded -- 1064 sequences are left sequence decoded -- 1063 sequences are left sequence decoded -- 1062 sequences are left sequence decoded -- 1061 sequences are left sequence decoded -- 1060 sequences are left sequence decoded -- 1059 sequences are left sequence decoded -- 1058 sequences are left sequence decoded -- 1057 sequences are left sequence decoded -- 1056 sequences are left sequence decoded -- 1055 sequences are left sequence decoded -- 1054 sequences are left sequence decoded -- 1053 sequences are left sequence decoded -- 1052 sequences are left sequence decoded -- 1051 sequences are left sequence decoded -- 1050 sequences are left sequence decoded -- 1049 sequences are left sequence decoded -- 1048 sequences are left sequence decoded -- 1047 sequences are left sequence decoded -- 1046 sequences are left sequence decoded -- 1045 sequences are left sequence decoded -- 1044 sequences are left sequence decoded -- 1043 sequences are left sequence decoded -- 1042 sequences are left sequence decoded -- 1041 sequences are left sequence decoded -- 1040 sequences are left sequence decoded -- 1039 sequences are left sequence decoded -- 1038 sequences are left sequence decoded -- 1037 sequences are left sequence decoded -- 1036 sequences are left sequence decoded -- 1035 sequences are left sequence decoded -- 1034 sequences are left sequence decoded -- 1033 sequences are left sequence decoded -- 1032 sequences are left sequence decoded -- 1031 sequences are left sequence decoded -- 1030 sequences are left sequence decoded -- 1029 sequences are left sequence decoded -- 1028 sequences are left sequence decoded -- 1027 sequences are left sequence decoded -- 1026 sequences are left sequence decoded -- 1025 sequences are left sequence decoded -- 1024 sequences are left sequence decoded -- 1023 sequences are left sequence decoded -- 1022 sequences are left sequence decoded -- 1021 sequences are left sequence decoded -- 1020 sequences are left sequence decoded -- 1019 sequences are left sequence decoded -- 1018 sequences are left sequence decoded -- 1017 sequences are left sequence decoded -- 1016 sequences are left sequence decoded -- 1015 sequences are left sequence decoded -- 1014 sequences are left sequence decoded -- 1013 sequences are left sequence decoded -- 1012 sequences are left sequence decoded -- 1011 sequences are left sequence decoded -- 1010 sequences are left sequence decoded -- 1009 sequences are left sequence decoded -- 1008 sequences are left sequence decoded -- 1007 sequences are left sequence decoded -- 1006 sequences are left sequence decoded -- 1005 sequences are left sequence decoded -- 1004 sequences are left sequence decoded -- 1003 sequences are left sequence decoded -- 1002 sequences are left sequence decoded -- 1001 sequences are left sequence decoded -- 1000 sequences are left sequence decoded -- 999 sequences are left sequence decoded -- 998 sequences are left sequence decoded -- 997 sequences are left sequence decoded -- 996 sequences are left sequence decoded -- 995 sequences are left sequence decoded -- 994 sequences are left sequence decoded -- 993 sequences are left sequence decoded -- 992 sequences are left sequence decoded -- 991 sequences are left sequence decoded -- 990 sequences are left sequence decoded -- 989 sequences are left sequence decoded -- 988 sequences are left sequence decoded -- 987 sequences are left sequence decoded -- 986 sequences are left sequence decoded -- 985 sequences are left sequence decoded -- 984 sequences are left sequence decoded -- 983 sequences are left sequence decoded -- 982 sequences are left sequence decoded -- 981 sequences are left sequence decoded -- 980 sequences are left sequence decoded -- 979 sequences are left sequence decoded -- 978 sequences are left sequence decoded -- 977 sequences are left sequence decoded -- 976 sequences are left sequence decoded -- 975 sequences are left sequence decoded -- 974 sequences are left sequence decoded -- 973 sequences are left sequence decoded -- 972 sequences are left sequence decoded -- 971 sequences are left sequence decoded -- 970 sequences are left sequence decoded -- 969 sequences are left sequence decoded -- 968 sequences are left sequence decoded -- 967 sequences are left sequence decoded -- 966 sequences are left sequence decoded -- 965 sequences are left sequence decoded -- 964 sequences are left sequence decoded -- 963 sequences are left sequence decoded -- 962 sequences are left sequence decoded -- 961 sequences are left sequence decoded -- 960 sequences are left sequence decoded -- 959 sequences are left sequence decoded -- 958 sequences are left sequence decoded -- 957 sequences are left sequence decoded -- 956 sequences are left sequence decoded -- 955 sequences are left sequence decoded -- 954 sequences are left sequence decoded -- 953 sequences are left sequence decoded -- 952 sequences are left sequence decoded -- 951 sequences are left sequence decoded -- 950 sequences are left sequence decoded -- 949 sequences are left sequence decoded -- 948 sequences are left sequence decoded -- 947 sequences are left sequence decoded -- 946 sequences are left sequence decoded -- 945 sequences are left sequence decoded -- 944 sequences are left sequence decoded -- 943 sequences are left sequence decoded -- 942 sequences are left sequence decoded -- 941 sequences are left sequence decoded -- 940 sequences are left sequence decoded -- 939 sequences are left sequence decoded -- 938 sequences are left sequence decoded -- 937 sequences are left sequence decoded -- 936 sequences are left sequence decoded -- 935 sequences are left sequence decoded -- 934 sequences are left sequence decoded -- 933 sequences are left sequence decoded -- 932 sequences are left sequence decoded -- 931 sequences are left sequence decoded -- 930 sequences are left sequence decoded -- 929 sequences are left sequence decoded -- 928 sequences are left sequence decoded -- 927 sequences are left sequence decoded -- 926 sequences are left sequence decoded -- 925 sequences are left sequence decoded -- 924 sequences are left sequence decoded -- 923 sequences are left sequence decoded -- 922 sequences are left sequence decoded -- 921 sequences are left sequence decoded -- 920 sequences are left sequence decoded -- 919 sequences are left sequence decoded -- 918 sequences are left sequence decoded -- 917 sequences are left sequence decoded -- 916 sequences are left sequence decoded -- 915 sequences are left sequence decoded -- 914 sequences are left sequence decoded -- 913 sequences are left sequence decoded -- 912 sequences are left sequence decoded -- 911 sequences are left sequence decoded -- 910 sequences are left sequence decoded -- 909 sequences are left sequence decoded -- 908 sequences are left sequence decoded -- 907 sequences are left sequence decoded -- 906 sequences are left sequence decoded -- 905 sequences are left sequence decoded -- 904 sequences are left sequence decoded -- 903 sequences are left sequence decoded -- 902 sequences are left sequence decoded -- 901 sequences are left sequence decoded -- 900 sequences are left sequence decoded -- 899 sequences are left sequence decoded -- 898 sequences are left sequence decoded -- 897 sequences are left sequence decoded -- 896 sequences are left sequence decoded -- 895 sequences are left sequence decoded -- 894 sequences are left sequence decoded -- 893 sequences are left sequence decoded -- 892 sequences are left sequence decoded -- 891 sequences are left sequence decoded -- 890 sequences are left sequence decoded -- 889 sequences are left sequence decoded -- 888 sequences are left sequence decoded -- 887 sequences are left sequence decoded -- 886 sequences are left sequence decoded -- 885 sequences are left sequence decoded -- 884 sequences are left sequence decoded -- 883 sequences are left sequence decoded -- 882 sequences are left sequence decoded -- 881 sequences are left sequence decoded -- 880 sequences are left sequence decoded -- 879 sequences are left sequence decoded -- 878 sequences are left sequence decoded -- 877 sequences are left sequence decoded -- 876 sequences are left sequence decoded -- 875 sequences are left sequence decoded -- 874 sequences are left sequence decoded -- 873 sequences are left sequence decoded -- 872 sequences are left sequence decoded -- 871 sequences are left sequence decoded -- 870 sequences are left sequence decoded -- 869 sequences are left sequence decoded -- 868 sequences are left sequence decoded -- 867 sequences are left sequence decoded -- 866 sequences are left sequence decoded -- 865 sequences are left sequence decoded -- 864 sequences are left sequence decoded -- 863 sequences are left sequence decoded -- 862 sequences are left sequence decoded -- 861 sequences are left sequence decoded -- 860 sequences are left sequence decoded -- 859 sequences are left sequence decoded -- 858 sequences are left sequence decoded -- 857 sequences are left sequence decoded -- 856 sequences are left sequence decoded -- 855 sequences are left sequence decoded -- 854 sequences are left sequence decoded -- 853 sequences are left sequence decoded -- 852 sequences are left sequence decoded -- 851 sequences are left sequence decoded -- 850 sequences are left sequence decoded -- 849 sequences are left sequence decoded -- 848 sequences are left sequence decoded -- 847 sequences are left sequence decoded -- 846 sequences are left sequence decoded -- 845 sequences are left sequence decoded -- 844 sequences are left sequence decoded -- 843 sequences are left sequence decoded -- 842 sequences are left sequence decoded -- 841 sequences are left sequence decoded -- 840 sequences are left sequence decoded -- 839 sequences are left sequence decoded -- 838 sequences are left sequence decoded -- 837 sequences are left sequence decoded -- 836 sequences are left sequence decoded -- 835 sequences are left sequence decoded -- 834 sequences are left sequence decoded -- 833 sequences are left sequence decoded -- 832 sequences are left sequence decoded -- 831 sequences are left sequence decoded -- 830 sequences are left sequence decoded -- 829 sequences are left sequence decoded -- 828 sequences are left sequence decoded -- 827 sequences are left sequence decoded -- 826 sequences are left sequence decoded -- 825 sequences are left sequence decoded -- 824 sequences are left sequence decoded -- 823 sequences are left sequence decoded -- 822 sequences are left sequence decoded -- 821 sequences are left sequence decoded -- 820 sequences are left sequence decoded -- 819 sequences are left sequence decoded -- 818 sequences are left sequence decoded -- 817 sequences are left sequence decoded -- 816 sequences are left sequence decoded -- 815 sequences are left sequence decoded -- 814 sequences are left sequence decoded -- 813 sequences are left sequence decoded -- 812 sequences are left sequence decoded -- 811 sequences are left sequence decoded -- 810 sequences are left sequence decoded -- 809 sequences are left sequence decoded -- 808 sequences are left sequence decoded -- 807 sequences are left sequence decoded -- 806 sequences are left sequence decoded -- 805 sequences are left sequence decoded -- 804 sequences are left sequence decoded -- 803 sequences are left sequence decoded -- 802 sequences are left sequence decoded -- 801 sequences are left sequence decoded -- 800 sequences are left sequence decoded -- 799 sequences are left sequence decoded -- 798 sequences are left sequence decoded -- 797 sequences are left sequence decoded -- 796 sequences are left sequence decoded -- 795 sequences are left sequence decoded -- 794 sequences are left sequence decoded -- 793 sequences are left sequence decoded -- 792 sequences are left sequence decoded -- 791 sequences are left sequence decoded -- 790 sequences are left sequence decoded -- 789 sequences are left sequence decoded -- 788 sequences are left sequence decoded -- 787 sequences are left sequence decoded -- 786 sequences are left sequence decoded -- 785 sequences are left sequence decoded -- 784 sequences are left sequence decoded -- 783 sequences are left sequence decoded -- 782 sequences are left sequence decoded -- 781 sequences are left sequence decoded -- 780 sequences are left sequence decoded -- 779 sequences are left sequence decoded -- 778 sequences are left sequence decoded -- 777 sequences are left sequence decoded -- 776 sequences are left sequence decoded -- 775 sequences are left sequence decoded -- 774 sequences are left sequence decoded -- 773 sequences are left sequence decoded -- 772 sequences are left sequence decoded -- 771 sequences are left sequence decoded -- 770 sequences are left sequence decoded -- 769 sequences are left sequence decoded -- 768 sequences are left sequence decoded -- 767 sequences are left sequence decoded -- 766 sequences are left sequence decoded -- 765 sequences are left sequence decoded -- 764 sequences are left sequence decoded -- 763 sequences are left sequence decoded -- 762 sequences are left sequence decoded -- 761 sequences are left sequence decoded -- 760 sequences are left sequence decoded -- 759 sequences are left sequence decoded -- 758 sequences are left sequence decoded -- 757 sequences are left sequence decoded -- 756 sequences are left sequence decoded -- 755 sequences are left sequence decoded -- 754 sequences are left sequence decoded -- 753 sequences are left sequence decoded -- 752 sequences are left sequence decoded -- 751 sequences are left sequence decoded -- 750 sequences are left sequence decoded -- 749 sequences are left sequence decoded -- 748 sequences are left sequence decoded -- 747 sequences are left sequence decoded -- 746 sequences are left sequence decoded -- 745 sequences are left sequence decoded -- 744 sequences are left sequence decoded -- 743 sequences are left sequence decoded -- 742 sequences are left sequence decoded -- 741 sequences are left sequence decoded -- 740 sequences are left sequence decoded -- 739 sequences are left sequence decoded -- 738 sequences are left sequence decoded -- 737 sequences are left sequence decoded -- 736 sequences are left sequence decoded -- 735 sequences are left sequence decoded -- 734 sequences are left sequence decoded -- 733 sequences are left sequence decoded -- 732 sequences are left sequence decoded -- 731 sequences are left sequence decoded -- 730 sequences are left sequence decoded -- 729 sequences are left sequence decoded -- 728 sequences are left sequence decoded -- 727 sequences are left sequence decoded -- 726 sequences are left sequence decoded -- 725 sequences are left sequence decoded -- 724 sequences are left sequence decoded -- 723 sequences are left sequence decoded -- 722 sequences are left sequence decoded -- 721 sequences are left sequence decoded -- 720 sequences are left sequence decoded -- 719 sequences are left sequence decoded -- 718 sequences are left sequence decoded -- 717 sequences are left sequence decoded -- 716 sequences are left sequence decoded -- 715 sequences are left sequence decoded -- 714 sequences are left sequence decoded -- 713 sequences are left sequence decoded -- 712 sequences are left sequence decoded -- 711 sequences are left sequence decoded -- 710 sequences are left sequence decoded -- 709 sequences are left sequence decoded -- 708 sequences are left sequence decoded -- 707 sequences are left sequence decoded -- 706 sequences are left sequence decoded -- 705 sequences are left sequence decoded -- 704 sequences are left sequence decoded -- 703 sequences are left sequence decoded -- 702 sequences are left sequence decoded -- 701 sequences are left sequence decoded -- 700 sequences are left sequence decoded -- 699 sequences are left sequence decoded -- 698 sequences are left sequence decoded -- 697 sequences are left sequence decoded -- 696 sequences are left sequence decoded -- 695 sequences are left sequence decoded -- 694 sequences are left sequence decoded -- 693 sequences are left sequence decoded -- 692 sequences are left sequence decoded -- 691 sequences are left sequence decoded -- 690 sequences are left sequence decoded -- 689 sequences are left sequence decoded -- 688 sequences are left sequence decoded -- 687 sequences are left sequence decoded -- 686 sequences are left sequence decoded -- 685 sequences are left sequence decoded -- 684 sequences are left sequence decoded -- 683 sequences are left sequence decoded -- 682 sequences are left sequence decoded -- 681 sequences are left sequence decoded -- 680 sequences are left sequence decoded -- 679 sequences are left sequence decoded -- 678 sequences are left sequence decoded -- 677 sequences are left sequence decoded -- 676 sequences are left sequence decoded -- 675 sequences are left sequence decoded -- 674 sequences are left sequence decoded -- 673 sequences are left sequence decoded -- 672 sequences are left sequence decoded -- 671 sequences are left sequence decoded -- 670 sequences are left sequence decoded -- 669 sequences are left sequence decoded -- 668 sequences are left sequence decoded -- 667 sequences are left sequence decoded -- 666 sequences are left sequence decoded -- 665 sequences are left sequence decoded -- 664 sequences are left sequence decoded -- 663 sequences are left sequence decoded -- 662 sequences are left sequence decoded -- 661 sequences are left sequence decoded -- 660 sequences are left sequence decoded -- 659 sequences are left sequence decoded -- 658 sequences are left sequence decoded -- 657 sequences are left sequence decoded -- 656 sequences are left sequence decoded -- 655 sequences are left sequence decoded -- 654 sequences are left sequence decoded -- 653 sequences are left sequence decoded -- 652 sequences are left sequence decoded -- 651 sequences are left sequence decoded -- 650 sequences are left sequence decoded -- 649 sequences are left sequence decoded -- 648 sequences are left sequence decoded -- 647 sequences are left sequence decoded -- 646 sequences are left sequence decoded -- 645 sequences are left sequence decoded -- 644 sequences are left sequence decoded -- 643 sequences are left sequence decoded -- 642 sequences are left sequence decoded -- 641 sequences are left sequence decoded -- 640 sequences are left sequence decoded -- 639 sequences are left sequence decoded -- 638 sequences are left sequence decoded -- 637 sequences are left sequence decoded -- 636 sequences are left sequence decoded -- 635 sequences are left sequence decoded -- 634 sequences are left sequence decoded -- 633 sequences are left sequence decoded -- 632 sequences are left sequence decoded -- 631 sequences are left sequence decoded -- 630 sequences are left sequence decoded -- 629 sequences are left sequence decoded -- 628 sequences are left sequence decoded -- 627 sequences are left sequence decoded -- 626 sequences are left sequence decoded -- 625 sequences are left sequence decoded -- 624 sequences are left sequence decoded -- 623 sequences are left sequence decoded -- 622 sequences are left sequence decoded -- 621 sequences are left sequence decoded -- 620 sequences are left sequence decoded -- 619 sequences are left sequence decoded -- 618 sequences are left sequence decoded -- 617 sequences are left sequence decoded -- 616 sequences are left sequence decoded -- 615 sequences are left sequence decoded -- 614 sequences are left sequence decoded -- 613 sequences are left sequence decoded -- 612 sequences are left sequence decoded -- 611 sequences are left sequence decoded -- 610 sequences are left sequence decoded -- 609 sequences are left sequence decoded -- 608 sequences are left sequence decoded -- 607 sequences are left sequence decoded -- 606 sequences are left sequence decoded -- 605 sequences are left sequence decoded -- 604 sequences are left sequence decoded -- 603 sequences are left sequence decoded -- 602 sequences are left sequence decoded -- 601 sequences are left sequence decoded -- 600 sequences are left sequence decoded -- 599 sequences are left sequence decoded -- 598 sequences are left sequence decoded -- 597 sequences are left sequence decoded -- 596 sequences are left sequence decoded -- 595 sequences are left sequence decoded -- 594 sequences are left sequence decoded -- 593 sequences are left sequence decoded -- 592 sequences are left sequence decoded -- 591 sequences are left sequence decoded -- 590 sequences are left sequence decoded -- 589 sequences are left sequence decoded -- 588 sequences are left sequence decoded -- 587 sequences are left sequence decoded -- 586 sequences are left sequence decoded -- 585 sequences are left sequence decoded -- 584 sequences are left sequence decoded -- 583 sequences are left sequence decoded -- 582 sequences are left sequence decoded -- 581 sequences are left sequence decoded -- 580 sequences are left sequence decoded -- 579 sequences are left sequence decoded -- 578 sequences are left sequence decoded -- 577 sequences are left sequence decoded -- 576 sequences are left sequence decoded -- 575 sequences are left sequence decoded -- 574 sequences are left sequence decoded -- 573 sequences are left sequence decoded -- 572 sequences are left sequence decoded -- 571 sequences are left sequence decoded -- 570 sequences are left sequence decoded -- 569 sequences are left sequence decoded -- 568 sequences are left sequence decoded -- 567 sequences are left sequence decoded -- 566 sequences are left sequence decoded -- 565 sequences are left sequence decoded -- 564 sequences are left sequence decoded -- 563 sequences are left sequence decoded -- 562 sequences are left sequence decoded -- 561 sequences are left sequence decoded -- 560 sequences are left sequence decoded -- 559 sequences are left sequence decoded -- 558 sequences are left sequence decoded -- 557 sequences are left sequence decoded -- 556 sequences are left sequence decoded -- 555 sequences are left sequence decoded -- 554 sequences are left sequence decoded -- 553 sequences are left sequence decoded -- 552 sequences are left sequence decoded -- 551 sequences are left sequence decoded -- 550 sequences are left sequence decoded -- 549 sequences are left sequence decoded -- 548 sequences are left sequence decoded -- 547 sequences are left sequence decoded -- 546 sequences are left sequence decoded -- 545 sequences are left sequence decoded -- 544 sequences are left sequence decoded -- 543 sequences are left sequence decoded -- 542 sequences are left sequence decoded -- 541 sequences are left sequence decoded -- 540 sequences are left sequence decoded -- 539 sequences are left sequence decoded -- 538 sequences are left sequence decoded -- 537 sequences are left sequence decoded -- 536 sequences are left sequence decoded -- 535 sequences are left sequence decoded -- 534 sequences are left sequence decoded -- 533 sequences are left sequence decoded -- 532 sequences are left sequence decoded -- 531 sequences are left sequence decoded -- 530 sequences are left sequence decoded -- 529 sequences are left sequence decoded -- 528 sequences are left sequence decoded -- 527 sequences are left sequence decoded -- 526 sequences are left sequence decoded -- 525 sequences are left sequence decoded -- 524 sequences are left sequence decoded -- 523 sequences are left sequence decoded -- 522 sequences are left sequence decoded -- 521 sequences are left sequence decoded -- 520 sequences are left sequence decoded -- 519 sequences are left sequence decoded -- 518 sequences are left sequence decoded -- 517 sequences are left sequence decoded -- 516 sequences are left sequence decoded -- 515 sequences are left sequence decoded -- 514 sequences are left sequence decoded -- 513 sequences are left sequence decoded -- 512 sequences are left sequence decoded -- 511 sequences are left sequence decoded -- 510 sequences are left sequence decoded -- 509 sequences are left sequence decoded -- 508 sequences are left sequence decoded -- 507 sequences are left sequence decoded -- 506 sequences are left sequence decoded -- 505 sequences are left sequence decoded -- 504 sequences are left sequence decoded -- 503 sequences are left sequence decoded -- 502 sequences are left sequence decoded -- 501 sequences are left sequence decoded -- 500 sequences are left sequence decoded -- 499 sequences are left sequence decoded -- 498 sequences are left sequence decoded -- 497 sequences are left sequence decoded -- 496 sequences are left sequence decoded -- 495 sequences are left sequence decoded -- 494 sequences are left sequence decoded -- 493 sequences are left sequence decoded -- 492 sequences are left sequence decoded -- 491 sequences are left sequence decoded -- 490 sequences are left sequence decoded -- 489 sequences are left sequence decoded -- 488 sequences are left sequence decoded -- 487 sequences are left sequence decoded -- 486 sequences are left sequence decoded -- 485 sequences are left sequence decoded -- 484 sequences are left sequence decoded -- 483 sequences are left sequence decoded -- 482 sequences are left sequence decoded -- 481 sequences are left sequence decoded -- 480 sequences are left sequence decoded -- 479 sequences are left sequence decoded -- 478 sequences are left sequence decoded -- 477 sequences are left sequence decoded -- 476 sequences are left sequence decoded -- 475 sequences are left sequence decoded -- 474 sequences are left sequence decoded -- 473 sequences are left sequence decoded -- 472 sequences are left sequence decoded -- 471 sequences are left sequence decoded -- 470 sequences are left sequence decoded -- 469 sequences are left sequence decoded -- 468 sequences are left sequence decoded -- 467 sequences are left sequence decoded -- 466 sequences are left sequence decoded -- 465 sequences are left sequence decoded -- 464 sequences are left sequence decoded -- 463 sequences are left sequence decoded -- 462 sequences are left sequence decoded -- 461 sequences are left sequence decoded -- 460 sequences are left sequence decoded -- 459 sequences are left sequence decoded -- 458 sequences are left sequence decoded -- 457 sequences are left sequence decoded -- 456 sequences are left sequence decoded -- 455 sequences are left sequence decoded -- 454 sequences are left sequence decoded -- 453 sequences are left sequence decoded -- 452 sequences are left sequence decoded -- 451 sequences are left sequence decoded -- 450 sequences are left sequence decoded -- 449 sequences are left sequence decoded -- 448 sequences are left sequence decoded -- 447 sequences are left sequence decoded -- 446 sequences are left sequence decoded -- 445 sequences are left sequence decoded -- 444 sequences are left sequence decoded -- 443 sequences are left sequence decoded -- 442 sequences are left sequence decoded -- 441 sequences are left sequence decoded -- 440 sequences are left sequence decoded -- 439 sequences are left sequence decoded -- 438 sequences are left sequence decoded -- 437 sequences are left sequence decoded -- 436 sequences are left sequence decoded -- 435 sequences are left sequence decoded -- 434 sequences are left sequence decoded -- 433 sequences are left sequence decoded -- 432 sequences are left sequence decoded -- 431 sequences are left sequence decoded -- 430 sequences are left sequence decoded -- 429 sequences are left sequence decoded -- 428 sequences are left sequence decoded -- 427 sequences are left sequence decoded -- 426 sequences are left sequence decoded -- 425 sequences are left sequence decoded -- 424 sequences are left sequence decoded -- 423 sequences are left sequence decoded -- 422 sequences are left sequence decoded -- 421 sequences are left sequence decoded -- 420 sequences are left sequence decoded -- 419 sequences are left sequence decoded -- 418 sequences are left sequence decoded -- 417 sequences are left sequence decoded -- 416 sequences are left sequence decoded -- 415 sequences are left sequence decoded -- 414 sequences are left sequence decoded -- 413 sequences are left sequence decoded -- 412 sequences are left sequence decoded -- 411 sequences are left sequence decoded -- 410 sequences are left sequence decoded -- 409 sequences are left sequence decoded -- 408 sequences are left sequence decoded -- 407 sequences are left sequence decoded -- 406 sequences are left sequence decoded -- 405 sequences are left sequence decoded -- 404 sequences are left sequence decoded -- 403 sequences are left sequence decoded -- 402 sequences are left sequence decoded -- 401 sequences are left sequence decoded -- 400 sequences are left sequence decoded -- 399 sequences are left sequence decoded -- 398 sequences are left sequence decoded -- 397 sequences are left sequence decoded -- 396 sequences are left sequence decoded -- 395 sequences are left sequence decoded -- 394 sequences are left sequence decoded -- 393 sequences are left sequence decoded -- 392 sequences are left sequence decoded -- 391 sequences are left sequence decoded -- 390 sequences are left sequence decoded -- 389 sequences are left sequence decoded -- 388 sequences are left sequence decoded -- 387 sequences are left sequence decoded -- 386 sequences are left sequence decoded -- 385 sequences are left sequence decoded -- 384 sequences are left sequence decoded -- 383 sequences are left sequence decoded -- 382 sequences are left sequence decoded -- 381 sequences are left sequence decoded -- 380 sequences are left sequence decoded -- 379 sequences are left sequence decoded -- 378 sequences are left sequence decoded -- 377 sequences are left sequence decoded -- 376 sequences are left sequence decoded -- 375 sequences are left sequence decoded -- 374 sequences are left sequence decoded -- 373 sequences are left sequence decoded -- 372 sequences are left sequence decoded -- 371 sequences are left sequence decoded -- 370 sequences are left sequence decoded -- 369 sequences are left sequence decoded -- 368 sequences are left sequence decoded -- 367 sequences are left sequence decoded -- 366 sequences are left sequence decoded -- 365 sequences are left sequence decoded -- 364 sequences are left sequence decoded -- 363 sequences are left sequence decoded -- 362 sequences are left sequence decoded -- 361 sequences are left sequence decoded -- 360 sequences are left sequence decoded -- 359 sequences are left sequence decoded -- 358 sequences are left sequence decoded -- 357 sequences are left sequence decoded -- 356 sequences are left sequence decoded -- 355 sequences are left sequence decoded -- 354 sequences are left sequence decoded -- 353 sequences are left sequence decoded -- 352 sequences are left sequence decoded -- 351 sequences are left sequence decoded -- 350 sequences are left sequence decoded -- 349 sequences are left sequence decoded -- 348 sequences are left sequence decoded -- 347 sequences are left sequence decoded -- 346 sequences are left sequence decoded -- 345 sequences are left sequence decoded -- 344 sequences are left sequence decoded -- 343 sequences are left sequence decoded -- 342 sequences are left sequence decoded -- 341 sequences are left sequence decoded -- 340 sequences are left sequence decoded -- 339 sequences are left sequence decoded -- 338 sequences are left sequence decoded -- 337 sequences are left sequence decoded -- 336 sequences are left sequence decoded -- 335 sequences are left sequence decoded -- 334 sequences are left sequence decoded -- 333 sequences are left sequence decoded -- 332 sequences are left sequence decoded -- 331 sequences are left sequence decoded -- 330 sequences are left sequence decoded -- 329 sequences are left sequence decoded -- 328 sequences are left sequence decoded -- 327 sequences are left sequence decoded -- 326 sequences are left sequence decoded -- 325 sequences are left sequence decoded -- 324 sequences are left sequence decoded -- 323 sequences are left sequence decoded -- 322 sequences are left sequence decoded -- 321 sequences are left sequence decoded -- 320 sequences are left sequence decoded -- 319 sequences are left sequence decoded -- 318 sequences are left sequence decoded -- 317 sequences are left sequence decoded -- 316 sequences are left sequence decoded -- 315 sequences are left sequence decoded -- 314 sequences are left sequence decoded -- 313 sequences are left sequence decoded -- 312 sequences are left sequence decoded -- 311 sequences are left sequence decoded -- 310 sequences are left sequence decoded -- 309 sequences are left sequence decoded -- 308 sequences are left sequence decoded -- 307 sequences are left sequence decoded -- 306 sequences are left sequence decoded -- 305 sequences are left sequence decoded -- 304 sequences are left sequence decoded -- 303 sequences are left sequence decoded -- 302 sequences are left sequence decoded -- 301 sequences are left sequence decoded -- 300 sequences are left sequence decoded -- 299 sequences are left sequence decoded -- 298 sequences are left sequence decoded -- 297 sequences are left sequence decoded -- 296 sequences are left sequence decoded -- 295 sequences are left sequence decoded -- 294 sequences are left sequence decoded -- 293 sequences are left sequence decoded -- 292 sequences are left sequence decoded -- 291 sequences are left sequence decoded -- 290 sequences are left sequence decoded -- 289 sequences are left sequence decoded -- 288 sequences are left sequence decoded -- 287 sequences are left sequence decoded -- 286 sequences are left sequence decoded -- 285 sequences are left sequence decoded -- 284 sequences are left sequence decoded -- 283 sequences are left sequence decoded -- 282 sequences are left sequence decoded -- 281 sequences are left sequence decoded -- 280 sequences are left sequence decoded -- 279 sequences are left sequence decoded -- 278 sequences are left sequence decoded -- 277 sequences are left sequence decoded -- 276 sequences are left sequence decoded -- 275 sequences are left sequence decoded -- 274 sequences are left sequence decoded -- 273 sequences are left sequence decoded -- 272 sequences are left sequence decoded -- 271 sequences are left sequence decoded -- 270 sequences are left sequence decoded -- 269 sequences are left sequence decoded -- 268 sequences are left sequence decoded -- 267 sequences are left sequence decoded -- 266 sequences are left sequence decoded -- 265 sequences are left sequence decoded -- 264 sequences are left sequence decoded -- 263 sequences are left sequence decoded -- 262 sequences are left sequence decoded -- 261 sequences are left sequence decoded -- 260 sequences are left sequence decoded -- 259 sequences are left sequence decoded -- 258 sequences are left sequence decoded -- 257 sequences are left sequence decoded -- 256 sequences are left sequence decoded -- 255 sequences are left sequence decoded -- 254 sequences are left sequence decoded -- 253 sequences are left sequence decoded -- 252 sequences are left sequence decoded -- 251 sequences are left sequence decoded -- 250 sequences are left sequence decoded -- 249 sequences are left sequence decoded -- 248 sequences are left sequence decoded -- 247 sequences are left sequence decoded -- 246 sequences are left sequence decoded -- 245 sequences are left sequence decoded -- 244 sequences are left sequence decoded -- 243 sequences are left sequence decoded -- 242 sequences are left sequence decoded -- 241 sequences are left sequence decoded -- 240 sequences are left sequence decoded -- 239 sequences are left sequence decoded -- 238 sequences are left sequence decoded -- 237 sequences are left sequence decoded -- 236 sequences are left sequence decoded -- 235 sequences are left sequence decoded -- 234 sequences are left sequence decoded -- 233 sequences are left sequence decoded -- 232 sequences are left sequence decoded -- 231 sequences are left sequence decoded -- 230 sequences are left sequence decoded -- 229 sequences are left sequence decoded -- 228 sequences are left sequence decoded -- 227 sequences are left sequence decoded -- 226 sequences are left sequence decoded -- 225 sequences are left sequence decoded -- 224 sequences are left sequence decoded -- 223 sequences are left sequence decoded -- 222 sequences are left sequence decoded -- 221 sequences are left sequence decoded -- 220 sequences are left sequence decoded -- 219 sequences are left sequence decoded -- 218 sequences are left sequence decoded -- 217 sequences are left sequence decoded -- 216 sequences are left sequence decoded -- 215 sequences are left sequence decoded -- 214 sequences are left sequence decoded -- 213 sequences are left sequence decoded -- 212 sequences are left sequence decoded -- 211 sequences are left sequence decoded -- 210 sequences are left sequence decoded -- 209 sequences are left sequence decoded -- 208 sequences are left sequence decoded -- 207 sequences are left sequence decoded -- 206 sequences are left sequence decoded -- 205 sequences are left sequence decoded -- 204 sequences are left sequence decoded -- 203 sequences are left sequence decoded -- 202 sequences are left sequence decoded -- 201 sequences are left sequence decoded -- 200 sequences are left sequence decoded -- 199 sequences are left sequence decoded -- 198 sequences are left sequence decoded -- 197 sequences are left sequence decoded -- 196 sequences are left sequence decoded -- 195 sequences are left sequence decoded -- 194 sequences are left sequence decoded -- 193 sequences are left sequence decoded -- 192 sequences are left sequence decoded -- 191 sequences are left sequence decoded -- 190 sequences are left sequence decoded -- 189 sequences are left sequence decoded -- 188 sequences are left sequence decoded -- 187 sequences are left sequence decoded -- 186 sequences are left sequence decoded -- 185 sequences are left sequence decoded -- 184 sequences are left sequence decoded -- 183 sequences are left sequence decoded -- 182 sequences are left sequence decoded -- 181 sequences are left sequence decoded -- 180 sequences are left sequence decoded -- 179 sequences are left sequence decoded -- 178 sequences are left sequence decoded -- 177 sequences are left sequence decoded -- 176 sequences are left sequence decoded -- 175 sequences are left sequence decoded -- 174 sequences are left sequence decoded -- 173 sequences are left sequence decoded -- 172 sequences are left sequence decoded -- 171 sequences are left sequence decoded -- 170 sequences are left sequence decoded -- 169 sequences are left sequence decoded -- 168 sequences are left sequence decoded -- 167 sequences are left sequence decoded -- 166 sequences are left sequence decoded -- 165 sequences are left sequence decoded -- 164 sequences are left sequence decoded -- 163 sequences are left sequence decoded -- 162 sequences are left sequence decoded -- 161 sequences are left sequence decoded -- 160 sequences are left sequence decoded -- 159 sequences are left sequence decoded -- 158 sequences are left sequence decoded -- 157 sequences are left sequence decoded -- 156 sequences are left sequence decoded -- 155 sequences are left sequence decoded -- 154 sequences are left sequence decoded -- 153 sequences are left sequence decoded -- 152 sequences are left sequence decoded -- 151 sequences are left sequence decoded -- 150 sequences are left sequence decoded -- 149 sequences are left sequence decoded -- 148 sequences are left sequence decoded -- 147 sequences are left sequence decoded -- 146 sequences are left sequence decoded -- 145 sequences are left sequence decoded -- 144 sequences are left sequence decoded -- 143 sequences are left sequence decoded -- 142 sequences are left sequence decoded -- 141 sequences are left sequence decoded -- 140 sequences are left sequence decoded -- 139 sequences are left sequence decoded -- 138 sequences are left sequence decoded -- 137 sequences are left sequence decoded -- 136 sequences are left sequence decoded -- 135 sequences are left sequence decoded -- 134 sequences are left sequence decoded -- 133 sequences are left sequence decoded -- 132 sequences are left sequence decoded -- 131 sequences are left sequence decoded -- 130 sequences are left sequence decoded -- 129 sequences are left sequence decoded -- 128 sequences are left sequence decoded -- 127 sequences are left sequence decoded -- 126 sequences are left sequence decoded -- 125 sequences are left sequence decoded -- 124 sequences are left sequence decoded -- 123 sequences are left sequence decoded -- 122 sequences are left sequence decoded -- 121 sequences are left sequence decoded -- 120 sequences are left sequence decoded -- 119 sequences are left sequence decoded -- 118 sequences are left sequence decoded -- 117 sequences are left sequence decoded -- 116 sequences are left sequence decoded -- 115 sequences are left sequence decoded -- 114 sequences are left sequence decoded -- 113 sequences are left sequence decoded -- 112 sequences are left sequence decoded -- 111 sequences are left sequence decoded -- 110 sequences are left sequence decoded -- 109 sequences are left sequence decoded -- 108 sequences are left sequence decoded -- 107 sequences are left sequence decoded -- 106 sequences are left sequence decoded -- 105 sequences are left sequence decoded -- 104 sequences are left sequence decoded -- 103 sequences are left sequence decoded -- 102 sequences are left sequence decoded -- 101 sequences are left sequence decoded -- 100 sequences are left sequence decoded -- 99 sequences are left sequence decoded -- 98 sequences are left sequence decoded -- 97 sequences are left sequence decoded -- 96 sequences are left sequence decoded -- 95 sequences are left sequence decoded -- 94 sequences are left sequence decoded -- 93 sequences are left sequence decoded -- 92 sequences are left sequence decoded -- 91 sequences are left sequence decoded -- 90 sequences are left sequence decoded -- 89 sequences are left sequence decoded -- 88 sequences are left sequence decoded -- 87 sequences are left sequence decoded -- 86 sequences are left sequence decoded -- 85 sequences are left sequence decoded -- 84 sequences are left sequence decoded -- 83 sequences are left sequence decoded -- 82 sequences are left sequence decoded -- 81 sequences are left sequence decoded -- 80 sequences are left sequence decoded -- 79 sequences are left sequence decoded -- 78 sequences are left sequence decoded -- 77 sequences are left sequence decoded -- 76 sequences are left sequence decoded -- 75 sequences are left sequence decoded -- 74 sequences are left sequence decoded -- 73 sequences are left sequence decoded -- 72 sequences are left sequence decoded -- 71 sequences are left sequence decoded -- 70 sequences are left sequence decoded -- 69 sequences are left sequence decoded -- 68 sequences are left sequence decoded -- 67 sequences are left sequence decoded -- 66 sequences are left sequence decoded -- 65 sequences are left sequence decoded -- 64 sequences are left sequence decoded -- 63 sequences are left sequence decoded -- 62 sequences are left sequence decoded -- 61 sequences are left sequence decoded -- 60 sequences are left sequence decoded -- 59 sequences are left sequence decoded -- 58 sequences are left sequence decoded -- 57 sequences are left sequence decoded -- 56 sequences are left sequence decoded -- 55 sequences are left sequence decoded -- 54 sequences are left sequence decoded -- 53 sequences are left sequence decoded -- 52 sequences are left sequence decoded -- 51 sequences are left sequence decoded -- 50 sequences are left sequence decoded -- 49 sequences are left sequence decoded -- 48 sequences are left sequence decoded -- 47 sequences are left sequence decoded -- 46 sequences are left sequence decoded -- 45 sequences are left sequence decoded -- 44 sequences are left sequence decoded -- 43 sequences are left sequence decoded -- 42 sequences are left sequence decoded -- 41 sequences are left sequence decoded -- 40 sequences are left sequence decoded -- 39 sequences are left sequence decoded -- 38 sequences are left sequence decoded -- 37 sequences are left sequence decoded -- 36 sequences are left sequence decoded -- 35 sequences are left sequence decoded -- 34 sequences are left sequence decoded -- 33 sequences are left sequence decoded -- 32 sequences are left sequence decoded -- 31 sequences are left sequence decoded -- 30 sequences are left sequence decoded -- 29 sequences are left sequence decoded -- 28 sequences are left sequence decoded -- 27 sequences are left sequence decoded -- 26 sequences are left sequence decoded -- 25 sequences are left sequence decoded -- 24 sequences are left sequence decoded -- 23 sequences are left sequence decoded -- 22 sequences are left sequence decoded -- 21 sequences are left sequence decoded -- 20 sequences are left sequence decoded -- 19 sequences are left sequence decoded -- 18 sequences are left sequence decoded -- 17 sequences are left sequence decoded -- 16 sequences are left sequence decoded -- 15 sequences are left sequence decoded -- 14 sequences are left sequence decoded -- 13 sequences are left sequence decoded -- 12 sequences are left sequence decoded -- 11 sequences are left sequence decoded -- 10 sequences are left sequence decoded -- 9 sequences are left sequence decoded -- 8 sequences are left sequence decoded -- 7 sequences are left sequence decoded -- 6 sequences are left sequence decoded -- 5 sequences are left sequence decoded -- 4 sequences are left sequence decoded -- 3 sequences are left sequence decoded -- 2 sequences are left sequence decoded -- 1 sequences are left sequence decoded -- 0 sequences are left
4.2 Decoding performance¶
The decoded sequences will be found under the tutorials directory (i.e. current directory) under decoding_seqs directory.
|---tutorials | |---decoding_seqs | | |---tutorial_seqs_decoding.txtThe
tutorial_seqs_decoding.txt file will follow the same template/format of the test.txt file we parsed earlier, but this time with additional column containing our model's predictions.
We evaluate the decoding performance using two options:
In [12]:
# using conlleval.pl script to compute performance
new_decseqs_file = os.path.join(tutorials_dir, 'decoding_seqs','tutorial_seqs_decoding.txt')
# -r option specifies takes raw input -- no BIO/IOB representation
# -d option specifies the separator (in our case the file is tab delimited)
! ./conlleval.pl -r -d '\t' < $new_decseqs_file
processed 48396 tokens with 48396 phrases; found: 48396 phrases; correct: 47738.
accuracy: 98.64%; precision: 98.64%; recall: 98.64%; FB1: 98.64
'': precision: 77.78%; recall: 77.78%; FB1: 77.78 9
(: precision: 100.00%; recall: 100.00%; FB1: 100.00 649
): precision: 99.85%; recall: 100.00%; FB1: 99.92 652
,: precision: 99.88%; recall: 99.94%; FB1: 99.91 1713
.: precision: 100.00%; recall: 100.00%; FB1: 100.00 1877
:: precision: 98.13%; recall: 100.00%; FB1: 99.06 107
CC: precision: 99.55%; recall: 99.38%; FB1: 99.46 1765
CD: precision: 97.37%; recall: 95.28%; FB1: 96.31 228
DT: precision: 99.79%; recall: 99.76%; FB1: 99.77 3742
EX: precision: 90.48%; recall: 100.00%; FB1: 95.00 21
FW: precision: 98.22%; recall: 95.95%; FB1: 97.08 169
IN: precision: 99.80%; recall: 99.86%; FB1: 99.83 6356
JJ: precision: 97.15%; recall: 95.55%; FB1: 96.34 4835
JJR: precision: 92.11%; recall: 89.74%; FB1: 90.91 76
JJS: precision: 96.15%; recall: 92.59%; FB1: 94.34 26
LS: precision: 93.33%; recall: 100.00%; FB1: 96.55 15
MD: precision: 100.00%; recall: 100.00%; FB1: 100.00 179
NN: precision: 98.29%; recall: 99.02%; FB1: 98.65 14440
NNP: precision: 97.67%; recall: 95.45%; FB1: 96.55 43
NNPS: precision: 0.00%; recall: 0.00%; FB1: 0.00 0
NNS: precision: 98.82%; recall: 98.61%; FB1: 98.71 3294
PDT: precision: 66.67%; recall: 66.67%; FB1: 66.67 3
POS: precision: 94.12%; recall: 88.89%; FB1: 91.43 17
PRP: precision: 99.69%; recall: 100.00%; FB1: 99.84 321
PRP$: precision: 100.00%; recall: 100.00%; FB1: 100.00 145
RB: precision: 97.99%; recall: 97.99%; FB1: 97.99 1246
RBR: precision: 93.10%; recall: 84.38%; FB1: 88.52 29
RBS: precision: 81.82%; recall: 90.00%; FB1: 85.71 11
RP: precision: 66.67%; recall: 66.67%; FB1: 66.67 3
TO: precision: 100.00%; recall: 99.88%; FB1: 99.94 799
VB: precision: 98.45%; recall: 96.58%; FB1: 97.50 516
VBD: precision: 96.30%; recall: 96.20%; FB1: 96.25 999
VBG: precision: 95.29%; recall: 97.79%; FB1: 96.52 510
VBN: precision: 96.90%; recall: 97.02%; FB1: 96.96 1580
VBP: precision: 97.51%; recall: 98.52%; FB1: 98.01 682
VBZ: precision: 99.39%; recall: 97.80%; FB1: 98.59 982
WDT: precision: 99.65%; recall: 98.61%; FB1: 99.13 285
WP: precision: 100.00%; recall: 80.00%; FB1: 88.89 4
WP$: precision: 100.00%; recall: 100.00%; FB1: 100.00 7
WRB: precision: 96.15%; recall: 96.15%; FB1: 96.15 52
``: precision: 88.89%; recall: 88.89%; FB1: 88.89 9
In [16]:
from pyseqlab.crf_learning import SeqDecodingEvaluator
# initialize an evaluator
evaluator = SeqDecodingEvaluator(crf_m.model)
# evaluate performance
Y_seqs_dict = GenericTrainingWorkflow.map_pred_to_ref_seqs(seqs_decoded)
taglevel_perf = evaluator.compute_states_confmatrix(Y_seqs_dict)
perf = evaluator.get_performance_metric(taglevel_perf, "f1", exclude_states=[])
perf = evaluator.get_performance_metric(taglevel_perf, "precision", exclude_states=[])
perf = evaluator.get_performance_metric(taglevel_perf, "recall", exclude_states=[])
perf = evaluator.get_performance_metric(taglevel_perf, "accuracy", exclude_states=[])
print()
# demonstrate confusion matrix per label/state
for state, code in crf_percep.model.Y_codebook.items():
print("confusion_matrix for state= {} ".format(state))
print(taglevel_perf[code])
print("-"*40)
f1 0.9863289440577037 precision 0.9863289440577035 recall 0.9863289440577035 accuracy 0.999430372669071 confusion_matrix for state= VB [[ 4.00000000e+00 0.00000000e+00] [ 1.00000000e+00 4.76870000e+04]] ---------------------------------------- confusion_matrix for state= NNS [[ 0. 0.] [ 0. 47692.]] ---------------------------------------- confusion_matrix for state= , [[ 6.55000000e+02 1.70000000e+01] [ 1.00000000e+01 4.70100000e+04]] ---------------------------------------- confusion_matrix for state= . [[ 1.60000000e+01 1.00000000e+00] [ 2.00000000e+00 4.76730000e+04]] ---------------------------------------- confusion_matrix for state= FW [[ 1844. 0.] [ 0. 45848.]] ---------------------------------------- confusion_matrix for state= : [[ 2.81000000e+02 1.00000000e+00] [ 4.00000000e+00 4.74060000e+04]] ---------------------------------------- confusion_matrix for state= CD [[ 6.46000000e+02 1.00000000e+00] [ 0.00000000e+00 4.70450000e+04]] ---------------------------------------- confusion_matrix for state= IN [[ 9.53000000e+02 3.60000000e+01] [ 3.60000000e+01 4.66670000e+04]] ---------------------------------------- confusion_matrix for state= VBD [[ 644. 0.] [ 0. 47048.]] ---------------------------------------- confusion_matrix for state= MD [[ 1.40000000e+01 1.00000000e+00] [ 0.00000000e+00 4.76770000e+04]] ---------------------------------------- confusion_matrix for state= WRB [[ 7.00000000e+00 2.00000000e+00] [ 2.00000000e+00 4.76810000e+04]] ---------------------------------------- confusion_matrix for state= NNP [[ 7.90000000e+02 0.00000000e+00] [ 1.00000000e+00 4.69010000e+04]] ---------------------------------------- confusion_matrix for state= DT [[ 0. 0.] [ 0. 47692.]] ---------------------------------------- confusion_matrix for state= JJS [[ 1.80000000e+01 2.00000000e+00] [ 0.00000000e+00 4.76720000e+04]] ---------------------------------------- confusion_matrix for state= VBP [[ 2.00000000e+00 1.00000000e+00] [ 1.00000000e+00 4.76880000e+04]] ---------------------------------------- confusion_matrix for state= PRP [[ 0. 0.] [ 0. 47692.]] ---------------------------------------- confusion_matrix for state= JJ [[ 1.68500000e+03 2.00000000e+00] [ 1.00000000e+00 4.60040000e+04]] ---------------------------------------- confusion_matrix for state= ) [[ 1.20800000e+03 2.50000000e+01] [ 2.50000000e+01 4.64340000e+04]] ---------------------------------------- confusion_matrix for state= VBN [[ 8.00000000e+00 1.00000000e+00] [ 1.00000000e+00 4.76820000e+04]] ---------------------------------------- confusion_matrix for state= TO [[ 0. 0.] [ 0. 47692.]] ---------------------------------------- confusion_matrix for state= RB [[ 4618. 138.] [ 217. 42719.]] ---------------------------------------- confusion_matrix for state= RBS [[ 0. 0.] [ 0. 47692.]] ---------------------------------------- confusion_matrix for state= ( [[ 7.00000000e+00 0.00000000e+00] [ 0.00000000e+00 4.76850000e+04]] ---------------------------------------- confusion_matrix for state= PRP$ [[ 1.66000000e+02 3.00000000e+00] [ 7.00000000e+00 4.75160000e+04]] ---------------------------------------- confusion_matrix for state= NN [[ 3.17000000e+02 1.00000000e+00] [ 0.00000000e+00 4.73740000e+04]] ---------------------------------------- confusion_matrix for state= VBZ [[ 13989. 244.] [ 140. 33319.]] ---------------------------------------- confusion_matrix for state= EX [[ 4.20000000e+01 1.00000000e+00] [ 2.00000000e+00 4.76470000e+04]] ---------------------------------------- confusion_matrix for state= CC [[ 5.04000000e+02 8.00000000e+00] [ 1.80000000e+01 4.71620000e+04]] ---------------------------------------- confusion_matrix for state= VBG [[ 2.50000000e+01 2.00000000e+00] [ 5.00000000e+00 4.76600000e+04]] ---------------------------------------- confusion_matrix for state= WDT [[ 4.90000000e+01 2.00000000e+00] [ 2.00000000e+00 4.76390000e+04]] ----------------------------------------
The model's decoding performance when using beam search of size 5 is 98.63% F1 score.
Can we do better?! Well, exploring and experimenting with different feature templates, attributes and options for training a model are left as an exercise for the readers ... :)
5. Literature¶
Tsuruoka Y. et al. (2005) Developing a Robust Part-of-Speech Tagger for Biomedical Text. In: Bozanis P., Houstis E.N. (eds) Advances in Informatics. PCI 2005. Lecture Notes in Computer Science, vol 3746. Springer, Berlin, Heidelberg
Tateisi, Yuka and Jun'ichi Tsujii. Part-of-Speech Annotation of Biology Research Abstracts. In the Proceedings of 4th International Conference on Language Resource and Evaluation (LREC2004). IV. Lisbon, Portugal, pp. 1267-1270, May 2004.
In [ ]: