Command functions
Each command is also a function: from domainator.<command> import <command>. Its main input is its only positional parameter; the others are keyword-only and named like the command's options. Generated by scripts/docs/cli_reference.py; the entries are rendered by mkdocstrings.
Annotate and search sequences and structures
domainator.domainate.domainate(records, *, references, z, evalue=0.001, max_hits_per_cds=None, max_overlap=1, cpu=0, batch_size=10000, pre_parsed_references=None, best_annotation=False, gene_call=None, min_evalue=0.0, max_mode=False, foldseek=None, esm2_3di_weights=None, esm2_3di_device='cuda:0', overlap_by_db=False, max_hits_per_contig=None, foldseek_device='cpu', partial='include', foldseek_path=None, prostt5_weights=None, prostt5_device='cpu')
Annotate every input record with domain hits, retaining records without hits.
Search references against proteins/CDSs or nucleotide contigs, adding Domainator features passing evalue, min_evalue, overlap, and hit-count thresholds. references=None uses pre_parsed_references. z is HMMER's database size, cpu=0 uses all cores, and batch_size bounds the protein search batches. gene_call and partial control CDSs searched; max_mode disables heuristic filters. Foldseek options configure optional structural searches. best_annotation also adds Domain_Search best-hit features. Use domain_search for taxonomy filtering, hit-only output, or optional annotations.
domainator.domain_search.domain_search(targets, *, references, z, evalue=0.001, max_hits=None, max_overlap=1, cpu=0, batch_size=10000, add_annotations=False, cds_range=None, kb_range=None, whole_contig=False, normalize_direction=True, translate=False, gene_call=None, min_evalue=0.0, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude', alphabet='protein', max_mode=False, max_region_overlap=1.0, strand=None, decoys=None, max_hits_per_contig=None, partial='include', dedup='none')
Search references against targets in parallel and yield the hit regions.
The targets are searched in batches, each annotated as by domainate, and the region around each hit is extracted as by select_by_cds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
targets
|
Iterable[Union[SeqRecord, Tuple[str, int, int]]]
|
the sequences to search: SeqRecords (as parse_seqfiles reads them), or
(sequence file path, file offset, number of records) partitions of sequence files
(as offsets.i_partition_seqfiles makes them), but not both. Records are grouped into
batches of about |
required |
references
|
List[str]
|
references to search for: protein HMMs/FASTA, nucleotide HMMs/FASTA, or Infernal covariance models. |
required |
z
|
int
|
the database size for e-values (hmmsearch's Z), as |
required |
evalue
|
float
|
keep hits with an e-value at most this. |
0.001
|
max_hits
|
Optional[int]
|
keep only this many hits, the best-scoring. None: all. |
None
|
max_overlap
|
float
|
the largest fraction of a hit that may overlap a better hit on the same CDS. |
1
|
cpu
|
int
|
number of workers (0: one per physical core); values below 2 are raised to 2 (one merges the results, the others search). |
0
|
batch_size
|
int
|
about how many CDSs (or proteins) a worker searches at once. |
10000
|
add_annotations
|
bool
|
also add domain annotations (Domainator features) for retained hits, in addition to the search best-hit annotations (Domain_Search features). |
False
|
cds_range
|
Optional[Tuple[int, int]]
|
(upstream, downstream) numbers of CDSs to extract around each hit CDS. |
None
|
kb_range
|
Optional[Tuple[float, float]]
|
(upstream, downstream) kb to extract around each hit CDS. Partially enclosed features are truncated, with '<' or '>' marking the cut ends. |
None
|
whole_contig
|
bool
|
extract the whole contig of each hit (once, if it has several hits). |
False
|
normalize_direction
|
bool
|
reverse-complement regions whose hit CDS is on the reverse strand. |
True
|
translate
|
bool
|
emit peptides from matching CDSs in protein-reference searches. |
False
|
gene_call
|
Optional[str]
|
as for domainate. |
None
|
min_evalue
|
float
|
as for domainate. |
0.0
|
max_mode
|
bool
|
as for domainate. |
False
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
an NCBITaxonomy, needed by the taxonomy filters. |
None
|
include_taxids
|
Optional[Iterable[int]]
|
search proteins/CDSs or nucleotide hits in these taxids or descendants. |
None
|
exclude_taxids
|
Optional[Iterable[int]]
|
exclude these taxids or descendants, overriding inclusion. |
None
|
taxonomy_expr
|
Optional[str]
|
a boolean expression over taxids (operators & | ~ and parentheses), e.g. "2 & ~1224". Mutually exclusive with include_taxids/exclude_taxids. |
None
|
taxonomy_unknown
|
str
|
unresolved taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Protein/CDS targets are filtered before searching, nucleotide hits before ranking; expanded output is not re-filtered. |
'exclude'
|
alphabet
|
str
|
"protein" or "nucleotide", the molecule type of FASTA databases. |
'protein'
|
max_region_overlap
|
float
|
the largest fraction of overlap allowed between two output regions (1 or more: no filter). Regions are kept greedily in order of CDS start, if less than this fraction of them overlaps a region already kept. |
1.0
|
strand
|
Optional[str]
|
"f" or "r" to extract only regions around CDSs on that strand. None: both. |
None
|
decoys
|
Optional[Set[str]]
|
names of decoy domains: a CDS or protein whose best hit is a decoy is not a hit. |
None
|
max_hits_per_contig
|
Optional[int]
|
the most independent hits per contig. None: all. For contig-level nucleotide hits, each non-overlapping hit region is an independent hit. |
None
|
partial
|
str
|
which proteins/CDSs to search, by fragment status: "include" (all, the default), "exclude" (skip fragments), or "only" (only fragments). See domainate.domainate. |
'include'
|
dedup
|
str
|
output uniqueness policy: "none" (the default), "id", "sequence", or "sequence_rc". Records with the same ID must have the same sequence, so either sequence policy also ensures unique IDs. Sequence comparisons ignore case and terminal '*'; sequence_rc also matches nucleotide reverse complements. With max_hits, each worker and the parent retain at most max_hits winners, replacing copies with better-scoring hits, and sorting only the final winners. Equal scores favor earlier arrivals. Without max_hits, hits are yielded in input order, whatever cpu is, and the first representative in input order is kept. |
'none'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
Matching protein records, nucleotide hit regions/whole contigs, or translated |
SeqRecord
|
CDSs, according to the input and output options. Search best-hit feature |
SeqRecord
|
locations mark alignments, not necessarily the extent of the output sequence. |
domainator.find_features.find_features(records, *, tmbed=False, device='cuda:0', cpu=0, batch_size=4000, gene_call=None, model_dir=None, tmbed_model_dir=None, motifs=None, partial='include')
Annotate the CDSs (or proteins) of records with predicted features, yielding each record.
The features are TMbed's transmembrane regions and signal peptides, and matches to PROSITE patterns. With neither, the records are still processed, which normalizes CDS translations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterator[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
tmbed
|
bool
|
run the TMbed prediction. |
False
|
device
|
Optional[str]
|
the device to run TMbed on (None or "None": the CPU). |
'cuda:0'
|
cpu
|
int
|
number of CPU threads (0: one per physical core). |
0
|
batch_size
|
int
|
how many residues to embed at a time. |
4000
|
gene_call
|
Optional[str]
|
"all", "unannotated" or None, as for domainate. |
None
|
model_dir
|
Optional[str]
|
the directory for every algorithm's models (each can be overridden). |
None
|
tmbed_model_dir
|
Optional[str]
|
the directory for TMbed's ProtT5 model (overrides |
None
|
motifs
|
Optional[List[str]]
|
PROSITE patterns to search for, e.g. "CX(2)C". |
None
|
partial
|
str
|
which CDSs/proteins to annotate, by fragment status: "include" (all, the default), "exclude" (skip fragments), or "only" (only fragments). See partial.get_fragment_status. Records are yielded either way. |
'include'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
The annotated records. |
domainator.structure_domainate.structure_domainate(input_values, *, references, aligner, evalue=0.001, min_evalue=0.0, max_hits_per_cds=None, max_overlap=1.0, overlap_by_db=False, alignment_type=None, metrics=None, max_seqs=None, tmp_dir=None, keep_db=None, database_name=None, hits_tsv_out=None)
Yield annotated protein SeqRecords, one per input structure chain.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_values
|
List[str]
|
-i values: structure files, globs, directories, or a database prefix. |
required |
references
|
List[str]
|
-r values, same forms as input_values. |
required |
aligner
|
StructureAligner
|
a structure_lib.StructureAligner. |
required |
evalue
|
float
|
keep hits with E-value strictly below this. |
0.001
|
min_evalue
|
float
|
keep hits with E-value at or above this. |
0.0
|
max_hits_per_cds
|
Optional[int]
|
maximum annotations to add per record (None: all). |
None
|
max_overlap
|
float
|
maximum fractional overlap between kept annotations. >= 1 disables overlap filtering. |
1.0
|
overlap_by_db
|
bool
|
filter overlaps within each database rather than across all. |
False
|
alignment_type
|
Optional[int]
|
the foldseek alignment type (None: the backend's default). |
None
|
metrics
|
Optional[List[str]]
|
extra structural metrics to compute and store as qualifiers. |
None
|
max_seqs
|
Optional[int]
|
maximum hits the backend keeps per reference. |
None
|
tmp_dir
|
Optional[str]
|
parent directory for temporary databases and scratch files. |
None
|
keep_db
|
Optional[str]
|
persist the database built from the input structures at this prefix. |
None
|
database_name
|
Optional[str]
|
value for the /database qualifier. Defaults to the reference name. |
None
|
hits_tsv_out
|
Optional[str]
|
copy the backend's raw hit table here, for cross-checking. |
None
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
SeqRecord objects with molecule_type "protein". |
domainator.structure_search.structure_search(input_values, *, references, aligner, evalue=0.001, min_evalue=0.0, max_hits=None, max_overlap=1.0, overlap_by_db=False, add_annotations=False, alignment_type=None, metrics=None, max_seqs=None, tmp_dir=None, keep_db=None, database_name=None, hits_tsv_out=None)
Yield protein SeqRecords for database records that matched a reference.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_values
|
List[str]
|
-i values: the database to search. Structure files, globs, directories, or a prebuilt database prefix. |
required |
references
|
List[str]
|
-r values: the reference structures to search with. |
required |
aligner
|
StructureAligner
|
a structure_lib.StructureAligner. |
required |
evalue
|
float
|
keep hits with E-value strictly below this. |
0.001
|
min_evalue
|
float
|
keep hits with E-value at or above this. Useful for excluding a reference's hit to its own copy in the database. |
0.0
|
max_hits
|
Optional[int]
|
return at most this many records, the best-scoring ones. None means all. |
None
|
max_overlap
|
float
|
maximum fractional overlap between annotations on one record. |
1.0
|
overlap_by_db
|
bool
|
filter overlaps within each database rather than across all. |
False
|
add_annotations
|
bool
|
also write domain annotations (Domainator features) for retained hits, in addition to search best-hit annotations (Domain_Search features). |
False
|
alignment_type
|
Optional[int]
|
the foldseek alignment type (None: the backend's default). |
None
|
metrics
|
Optional[List[str]]
|
extra structural metrics to compute and store as qualifiers. |
None
|
max_seqs
|
Optional[int]
|
maximum hits the backend keeps per reference. |
None
|
tmp_dir
|
Optional[str]
|
parent directory for temporary databases and scratch files. |
None
|
keep_db
|
Optional[str]
|
persist the database built from input structures at this prefix. |
None
|
database_name
|
Optional[str]
|
value for the /database qualifier. Defaults to the reference name. |
None
|
hits_tsv_out
|
Optional[str]
|
copy the backend's raw hit table here, for cross-checking. |
None
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
Protein SeqRecords containing matching targets' full observed sequences, not |
SeqRecord
|
alignment crops. Search best-hit features mark the aligned regions. |
SeqRecord
|
Sorted best-first when max_hits is set, otherwise streamed in whatever order |
SeqRecord
|
the backend grouped them. |
Select, filter, and edit annotated records
domainator.clean_sequences.clean_sequences(records, *, alphabet='protein', clean_name=False, deduplicate_names=False, strip=False, filter_by_aa=False, stats=None)
Clean and filter sequences based on specified criteria.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
An iterable/iterator of SeqRecords |
required |
alphabet
|
str
|
"protein" or "nucleotide" - determines which canonical character set to use |
'protein'
|
clean_name
|
bool
|
If True, make names compatible with newick format |
False
|
deduplicate_names
|
bool
|
If True, ensure all sequence names are unique |
False
|
strip
|
bool
|
If True, strip non-canonical residues from sequence ends |
False
|
filter_by_aa
|
bool
|
If True, filter out sequences containing non-canonical residues |
False
|
stats
|
Optional[CleaningStats]
|
Optional CleaningStats object to track statistics |
None
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
Cleaned/filtered SeqRecords |
domainator.color_genbank.color_genbank(records, *, color_table=None, targets=frozenset({'cdss'}), search_hit_color=None, clear=False, color_table_out=None, databases=None, color_feature='name')
Yield records with color qualifiers on their domains and/or CDSs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
color_table
|
Optional[Dict[str, str]]
|
maps domain names (or database names, with |
None
|
targets
|
AbstractSet[str]
|
what to color: a set holding "domains", "cdss" (the default), or both. |
frozenset({'cdss'})
|
search_hit_color
|
Optional[str]
|
color the Domain_Search best-hit features this color. |
None
|
clear
|
bool
|
remove all existing colors first. |
False
|
color_table_out
|
Optional[str]
|
write the color table, with any colors assigned, to this path. |
None
|
databases
|
Optional[Set[str]]
|
color only by domains from these databases. None: all. |
None
|
color_feature
|
str
|
the qualifier that picks the color: "name" or "database". |
'name'
|
domainator.deduplicate_genbank.deduplicate_genbank(inputs, *, identity, algorithm='cd-hit', extra_args=None, bin_paths=None, add_count=None, cpu=0, alphabet='protein', both_strands=False, log_out_handle=sys.stderr, clusters=None)
Yield one representative record of each cluster of similar sequences.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inputs
|
List[str]
|
GenBank or FASTA files. They are read twice, so they can't be a stream. |
required |
identity
|
float
|
identity threshold, between 0 and 1. The "hash" algorithm only accepts 1. |
required |
algorithm
|
str
|
"cd-hit", "usearch", "diamond", or "hash" (exact matches). |
'cd-hit'
|
extra_args
|
Optional[List[str]]
|
additional command line arguments for the clustering program. |
None
|
bin_paths
|
Optional[Dict[str, str]]
|
None, or a dict of program name ("cd-hit", "cd-hit-est", "usearch", "diamond") to the path of its executable, for programs that are not on the PATH. |
None
|
add_count
|
Optional[str]
|
None, "prefix" or "suffix": add the cluster size to the representative's id. |
None
|
cpu
|
int
|
number of threads for the clustering program (0: one per physical core). |
0
|
alphabet
|
str
|
"protein" or "nucleotide", for FASTA inputs. |
'protein'
|
both_strands
|
bool
|
for nucleotide inputs, cluster each sequence with its reverse complement. |
False
|
log_out_handle
|
TextIO
|
where the clustering program's output, and the input and output sizes, go. |
stderr
|
clusters
|
Optional[List[Tuple[str, List[str]]]]
|
if a list, the (representative id, member ids) of every cluster are appended to it once the records have all been yielded. |
None
|
domainator.extract_domains.extract_domains(records, *, evalue=None, min_score=None, domains=None, extend_range=None, extend_up=None, extend_down=None, combine=False, keep_direction=False, keep_parent_id=False, annotation_type='domainator', databases=None, splice=False, partial='include', has_search_hit=False, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude')
Yield the regions of records covered by their domain annotations.
Output retains the input sequence type. Nucleotide regions are not translated, rounded to codons, or automatically expanded to whole CDSs. Combining/splicing operates per input record, not per CDS, and does not guarantee a coding sequence.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
evalue
|
Optional[float]
|
extract only domain/search-hit annotations with an e-value at most this, before extension, combination, or splicing. None: no cutoff. |
None
|
min_score
|
Optional[float]
|
extract only domain/search-hit annotations with a score at least this, before extension, combination, or splicing. None: no cutoff. |
None
|
domains
|
Optional[Set[str]]
|
select only domains with these names. None: all. |
None
|
extend_range
|
Optional[int]
|
the default of extend_up and extend_down (None: 0). |
None
|
extend_up
|
Optional[int]
|
extend upstream by this many amino acids for protein input or bases for nucleotide input. Not rounded to codons or restricted to CDS boundaries. None: extend_range. |
None
|
extend_down
|
Optional[int]
|
extend downstream in the same units as extend_up. None: extend_range. |
None
|
combine
|
bool
|
extract one enclosing span of all selected annotations in each input record, including intervening sequence, extended at the span's ends. Not grouped by CDS; may span different genes or strands. Raises an error for a circular record, whose enclosing region is ambiguous. |
False
|
keep_direction
|
bool
|
don't reverse-complement regions. By default, regions are flipped so
that the domain (with |
False
|
keep_parent_id
|
bool
|
use the parent record ID instead of a coordinate-based region ID. Applies to nucleotide and protein inputs and can produce duplicate IDs for different extracted sequences. |
False
|
annotation_type
|
str
|
which annotations to extract: "domainator", "domain_search", or "all". |
'domainator'
|
has_search_hit
|
bool
|
also extract search hits, ignoring domain names and databases but honoring evalue, min_score, and partial. Alone, extracts only search hits. |
False
|
databases
|
Optional[Set[str]]
|
select only domains from these databases. None: all. |
None
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
database required when taxonomy criteria are supplied. |
None
|
include_taxids
|
Optional[Iterable[int]]
|
include these taxids and descendants (OR). |
None
|
exclude_taxids
|
Optional[Iterable[int]]
|
exclude these taxids and descendants, overriding inclusion. |
None
|
taxonomy_expr
|
Optional[str]
|
boolean lineage expression, exclusive with include/exclude_taxids. |
None
|
taxonomy_unknown
|
str
|
unresolved taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Applied to the original domain location before extension/combine/splice; protein inputs use record taxonomy. |
'exclude'
|
splice
|
bool
|
when a contig has several selected domains, delete the sequence between them and join the domains, in the order they are read on the strand of the domain with the lowest start. Each annotation is extended before joining. Not grouped by CDS and not a guarantee of biological exon splicing or a valid coding sequence. |
False
|
partial
|
str
|
select domains by the fragment status of their CDS (or protein record): "include" (all, the default), "exclude" (skip domains on fragments), or "only" (only domains on fragments). Contig-level nucleotide hits are never on fragments. See partial.get_fragment_status. |
'include'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
The extracted regions. |
domainator.extract_peptides.extract_peptides(records, *, evalue=None, domains=None, cds_ids=None, domain_expr=None, keep_cds_feature=True, cds_id_type='name', annotation_type='domainator', strand=None, unannotated=False, _from_domain_search=False, extract_all=False, invert=False, _domain_search_negatives=None, databases=None, keep_parent_id=False, name_field=None, partial='include', has_search_hit=False, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude')
Extracts all peptide sequences from each SeqRecord in records. Yields the extracted peptides as SeqRecord objects. Args: records: an iterable of SeqRecords evalue: maximum e-value of domain/search-hit annotations used for CDS selection and carried onto the peptide record (None: no cutoff). domains: only extract peptides containing these domains domain_expr: also select CDSs satisfying this Boolean domain expression. cds_ids: only consider peptides from cdss with these names keep_cds_feature: if True, keep the CDS feature on the peptide record. If False, remove it. cds_id_type: if "name", then cds_ids is a list of CDS names. If "num", then cds_ids is a list of CDS numbers. annotation_type: which annotations supply domain names: "domainator", "domain_search", or "all". has_search_hit: also select CDSs with search hits passing evalue, ignoring domain names and databases. Participates in invert. strand: only extract peptides of CDSs on the specified strand. If None, then extract peptides from CDSs on both strands. forward strand is "f", reverse strand is "r". unannotated: if True, extract peptides from CDSs with no domain annotations. _from_domain_search: if True, then the input records are from a domain search, and we can skip all the annotation propagation. extract_all: select all CDSs before applying independent restrictions. With no selectors, this is implicit. invert: invert the combined CDS selectors, not strand, partial, or taxonomy. With no selectors, warn and ignore inversion. _domain_search_negatives: if not None, then this is a set of domain names that are considered negative hits. If a CDS has the best search hit as a negative, then it will be skipped. databases: if not None, then only consider domains from these databases when filtering by domain. default: all databases. keep_parent_id: use the parent contig ID instead of the resolved CDS name for peptide IDs. This can produce duplicate IDs for different peptide sequences. name_field: prefer this CDS qualifier for name matching and peptide IDs, falling back to protein_id, gene_id, locus_tag, then gene. partial: select CDSs by fragment status: "include" (default, all), "exclude" (skip fragments), or "only" (only fragments). A CDS is a fragment if its location has '<' or '>' (see utils.get_fragment_status). Not affected by invert. ncbi_taxonomy: database required when taxonomy criteria are supplied. include_taxids: include these taxids and descendants (OR). exclude_taxids: exclude these taxids and descendants, overriding inclusion. taxonomy_expr: boolean lineage expression, exclusive with include/exclude_taxids. taxonomy_unknown: unresolved taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Taxonomy is resolved on the original CDS, before translation, and is not affected by invert.
domainator.extract_unannotated.extract_unannotated(records, *, evalue=None, min_annotation_score=None, keep_direction=False, keep_parent_id=False, annotation_type='domainator', databases=None, largest=False, min_region_length=None, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude')
Yield the regions of the proteins in records that have no domain annotation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
protein SeqRecords (only proteins are supported). |
required |
evalue
|
Optional[float]
|
count only domain/search-hit annotations with an e-value at most this when finding unannotated gaps. None: no cutoff. |
None
|
min_annotation_score
|
Optional[float]
|
count only domain/search-hit annotations with a score at least this when finding unannotated gaps. Raising this can enlarge the gaps. None: no cutoff. |
None
|
keep_direction
|
bool
|
has no effect, since only proteins are supported; kept for compatibility with extract_domains. |
False
|
keep_parent_id
|
bool
|
use the parent protein ID instead of a coordinate-based region ID. This can produce duplicate IDs for different extracted sequences. |
False
|
annotation_type
|
str
|
which annotations count as annotated: "domainator", "domain_search", or "all". |
'domainator'
|
databases
|
Optional[Set[str]]
|
count only domains from these databases as annotations. None: all. |
None
|
largest
|
bool
|
yield only the largest unannotated region of each protein that is at least
|
False
|
min_region_length
|
Optional[int]
|
yield only unannotated gaps at least this many amino acids long, before choosing the largest gap. Not a cutoff on the input protein length. |
None
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
database required when taxonomy criteria are supplied. |
None
|
include_taxids
|
Optional[Iterable[int]]
|
include these taxids and descendants (OR). |
None
|
exclude_taxids
|
Optional[Iterable[int]]
|
exclude these taxids and descendants, overriding inclusion. |
None
|
taxonomy_expr
|
Optional[str]
|
boolean lineage expression, exclusive with include/exclude_taxids. |
None
|
taxonomy_unknown
|
str
|
unresolved taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Filters original protein records before extracting gaps. |
'exclude'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
The unannotated regions. |
domainator.filter_domains.filter_domains(records, *, evalue=None, max_overlap=1.0, keep_databases=None, databases=None, domains=None, annotation_type='all')
Yield records without the domain annotations that fail the filters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
evalue
|
Optional[float]
|
drop annotations with an e-value above this (None: no cutoff). |
None
|
max_overlap
|
float
|
the largest fraction of an annotation that may overlap a higher-scoring annotation on the same CDS (1: no overlap filter). |
1.0
|
keep_databases
|
Optional[Set[str]]
|
first drop the annotations from every other database. |
None
|
databases
|
Optional[Set[str]]
|
apply |
None
|
domains
|
Optional[Set[str]]
|
drop the filtered annotations whose names are not in this set. None: keep any name. |
None
|
annotation_type
|
str
|
which annotations to filter: "domainator", "domain_search" (the best hits written by domain_search), or "all". |
'all'
|
domainator.rotate_contigs.rotate_contigs(records, *, references=None, pre_parsed_references=None, z=1000, evalue=0.001, domains=None, databases=None, annotation_type='domainator', anchor='start', keep_direction=False, circularize=False, keep_name=False, keep_annotation=False, gene_call=None, cpu=0, max_mode=False, batch_size=1000, has_search_hit=False)
Rotate circular contigs so that their origin is set relative to a domain annotation.
Returns a generator of rotated SeqRecords. Contigs that can't be rotated are skipped with a warning. Arguments are validated when this function is called, before any records are read.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable
|
an iterable of nucleotide SeqRecords |
required |
references
|
Optional[List[str]]
|
hmm, cm, or fasta files to search against the records, to find new domains to rotate on. |
None
|
pre_parsed_references
|
Optional[Dict[str, Dict[str, Any]]]
|
the output of domainate.read_references, used instead of references. |
None
|
z
|
int
|
passed as the -Z parameter to hmmsearch. |
1000
|
evalue
|
float
|
the maximum evalue of new and pre-existing domains to rotate on. |
0.001
|
domains
|
Optional[Iterable[str]]
|
names of pre-existing annotations to rotate on, from annotation_type. |
None
|
databases
|
Optional[Iterable[str]]
|
only consider pre-existing annotations from these databases. If None, consider all databases. |
None
|
annotation_type
|
str
|
which pre-existing annotations to use: "domainator", "domain_search", or "all". With "domainator", specify domains to use pre-existing annotations. |
'domainator'
|
has_search_hit
|
bool
|
also consider search hits passing evalue, ignoring domain names and databases. The highest-ranked candidate still determines the anchor. |
False
|
anchor
|
str
|
"start", "end", or "center". Where to put the origin, relative to the domain in its own orientation. |
'start'
|
keep_direction
|
bool
|
if True, don't reverse complement contigs with the domain on the reverse strand. |
False
|
circularize
|
bool
|
if True, rotate linear contigs and fasta records as though they were circular. If False, skip them. |
False
|
keep_name
|
bool
|
if True, keep the original contig names. If False, rename contigs to note how they were rotated. |
False
|
keep_annotation
|
bool
|
if True, keep the new annotation of the domain that the contig was rotated on. Other new annotations are never kept. |
False
|
gene_call
|
Optional[str]
|
None, "all", or "unannotated". Passed to domainate to call genes with prodigal before the search. |
None
|
cpu
|
int
|
number of cpus to use for the search. 0 means use all available cpus. |
0
|
max_mode
|
bool
|
run the search in maximum sensitivity mode. |
False
|
batch_size
|
int
|
number of contigs to search at a time. |
1000
|
domainator.select_by_cds.select_by_cds(records, *, cds_ids=None, domains=None, domain_expr=None, cds_range=None, kb_range=None, whole_contig=False, normalize_direction=True, evalue=None, invert=False, annotation_type='domainator', max_region_overlap=1.0, strand=None, _from_domain_search=False, _domain_search_negatives=None, databases=None, unannotated=False, include_nucleic_acids=False, partial='include', has_search_hit=False, name_field=None, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude')
Yield the regions of records around the CDSs (or proteins) that are selected.
A CDS is selected if it matches any of cds_ids, domains, domain_expr,
unannotated and has_search_hit (or with none of them, every CDS), and then the strand and partial
filters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
cds_ids
|
Optional[Set[str]]
|
select the CDSs with these names. |
None
|
domains
|
Optional[Set[str]]
|
select the CDSs with a domain of one of these names. |
None
|
domain_expr
|
Optional[str]
|
select the CDSs whose domain names satisfy this boolean expression. |
None
|
cds_range
|
Optional[Tuple[int, int]]
|
(upstream, downstream) numbers of CDSs to extract around each selected CDS. |
None
|
kb_range
|
Optional[Tuple[float, float]]
|
(upstream, downstream) kb to extract around each selected CDS. Partially enclosed features are truncated, with '<' or '>' marking the cut ends; partially enclosed domain annotations are dropped. |
None
|
whole_contig
|
bool
|
extract the whole contig of each selected CDS (once, if it has several). |
False
|
normalize_direction
|
bool
|
reverse-complement regions whose selected CDS is on the reverse strand. |
True
|
evalue
|
Optional[float]
|
ignore domains with an e-value above this (None: no cutoff). |
None
|
invert
|
bool
|
invert the combined |
False
|
annotation_type
|
str
|
which annotations supply domain names: "domainator", "domain_search", or "all". Does not itself select CDSs. |
'domainator'
|
has_search_hit
|
bool
|
also select CDSs marked as domain_search hits passing evalue, ignoring domain names, expressions, and databases. Participates in invert and enables contig-level nucleotide search hits. |
False
|
max_region_overlap
|
float
|
the largest fraction of overlap allowed between two output regions (1 or more: no filter). |
1.0
|
strand
|
Optional[str]
|
"f" or "r" to select only CDSs on that strand. None: both. |
None
|
name_field
|
Optional[str]
|
prefer this CDS qualifier for name matching, falling back to gene_id, locus_tag, gene, then protein_id. Region IDs remain coordinate-based. |
None
|
_from_domain_search
|
bool
|
internal (domain_search): select the CDSs recorded as hits on the record. |
False
|
_domain_search_negatives
|
Optional[Set[str]]
|
internal (domain_search): with _from_domain_search, domain names to select against. |
None
|
databases
|
Optional[Set[str]]
|
consider only domains from these databases. None: all. |
None
|
unannotated
|
bool
|
select the CDSs with no domain annotations. |
False
|
include_nucleic_acids
|
bool
|
treat nucleotide (contig-level) annotations as CDSs, so they can be selected too. |
False
|
partial
|
str
|
select CDSs by fragment status: "include" (all, the default), "exclude" (skip fragments), or "only" (only fragments). A CDS is a fragment if its location has '<' or '>' (see partial.get_fragment_status). Not affected by invert. |
'include'
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
database required when taxonomy criteria are supplied. |
None
|
include_taxids
|
Optional[Iterable[int]]
|
include these taxids and descendants (OR). |
None
|
exclude_taxids
|
Optional[Iterable[int]]
|
exclude these taxids and descendants, overriding inclusion. |
None
|
taxonomy_expr
|
Optional[str]
|
boolean lineage expression, exclusive with include/exclude_taxids. |
None
|
taxonomy_unknown
|
str
|
unresolved taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Applies to the focus CDS/hit, not the expanded neighborhood; not affected by invert. |
'exclude'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
SeqRecords of the selected regions. |
domainator.select_by_contig.select_by_contig(records, *, domains=None, domain_expr=None, evalue=None, max_records=None, min_length=None, max_length=None, min_cds=None, max_cds=None, definition_regex=None, sequence_regex=None, contigs=None, contigs_regex=None, invert=False, ncbi_taxonomy=None, include_taxids=None, exclude_taxids=None, taxonomy_expr=None, taxonomy_unknown='exclude', databases=None, unannotated=False, min_search_score=None, max_search_score=None, annotation_type='domainator', partial='include', has_search_hit=False)
Yield the records of records that are selected.
A record is selected if it matches any of the name, definition, sequence, domain and
search-hit or search-score criteria (or with none of them, every record), and it must also pass the
length, CDS-count, taxonomy and partial filters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
domains
|
Optional[Set[str]]
|
select records with a domain of one of these names. |
None
|
domain_expr
|
Optional[str]
|
select records whose domain names satisfy this boolean expression. |
None
|
evalue
|
Optional[float]
|
consider only domains with an e-value at most this (None: no cutoff). |
None
|
max_records
|
Optional[int]
|
yield at most this many records. |
None
|
min_length
|
Optional[int]
|
keep only input records at least this long, in bp for nucleotide input or amino acids for protein input. Not affected by invert. |
None
|
max_length
|
Optional[int]
|
keep only input records at most this long, in bp for nucleotide input or amino acids for protein input. Not affected by invert. |
None
|
min_cds
|
Optional[int]
|
keep only records with at least this many CDSs. |
None
|
max_cds
|
Optional[int]
|
keep only records with at most this many CDSs. |
None
|
definition_regex
|
Optional[str]
|
select records whose definition matches this regular expression. |
None
|
sequence_regex
|
Optional[str]
|
select records whose sequence matches this regular expression. |
None
|
contigs
|
Optional[Set[str]]
|
select records with these names. |
None
|
contigs_regex
|
Optional[str]
|
select records whose names match this regular expression. |
None
|
invert
|
bool
|
invert the selection criteria (not the filters). |
False
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
an NCBITaxonomy, needed by the taxonomy filters. |
None
|
include_taxids
|
Optional[Iterable[int]]
|
keep only records with a taxid in this list (or below it). |
None
|
exclude_taxids
|
Optional[Iterable[int]]
|
drop records with a taxid in this list (or below it). |
None
|
taxonomy_expr
|
Optional[str]
|
a boolean expression over taxids (operators & | ~ and parentheses), e.g. "2 & ~1224". Mutually exclusive with include_taxids/exclude_taxids. |
None
|
taxonomy_unknown
|
str
|
unresolved record taxonomy policy: exclude (default), include, evaluate against an empty lineage, or error. Not affected by invert. |
'exclude'
|
databases
|
Optional[Set[str]]
|
consider only domains from these databases. None: all. |
None
|
unannotated
|
bool
|
select records with no domain annotations. |
False
|
min_search_score
|
Optional[float]
|
also select records whose existing Domain_Search annotation passes evalue and databases and scores at least this. Combined with max_search_score as an inclusive range, OR with other selectors, and inverted by invert. Also restricts has_search_hit scoring, which ignores databases. |
None
|
max_search_score
|
Optional[float]
|
upper bound of the inclusive search-score selector range. Not an independent restriction on records selected by other criteria. |
None
|
annotation_type
|
str
|
which annotations count as domains: "domainator", "domain_search", or "all". |
'domainator'
|
has_search_hit
|
bool
|
also select records with search hits passing evalue and search-score cutoffs, ignoring domain names, expressions, and databases. Participates in invert. |
False
|
partial
|
str
|
"include" (the default) keeps all records, "exclude" drops fragments, "only" keeps only fragments. A protein record is a fragment if partial.get_fragment_status says so (e.g. a UniProt " (Fragment)" header); a nucleotide contig is a fragment if it has a CDS with '<' or '>' in its location. |
'include'
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
The selected records. |
domainator.select_by_coord.select_by_coord(records, *, coordinate_selections)
Yield the regions of records that coordinate_selections lists.
CoordinateSelection rows, as load_coordinate_table reads them from a
table of contig, start, end (1-based, inclusive) and optionally strand and file. Each row that names a record gives one region, named like the regions of the other tools (model.region_id); a reverse-strand region is reverse complemented.
domainator.sort_contigs.sort_contigs(records, *, max_hits=None, domains=None, databases=None)
Yield records sorted by their best score, highest first, keeping the best max_hits.
The score is the best Domain_Search hit's, as domain_search orders its output, or with
domains the best score among Domainator domains with those names (and, with
databases, from those databases). Records with equal scores keep their input order.
Raises ValueError for a record with nothing to score.
domainator.trim_contigs.trim_contigs(records, *, kb_range=None, cds_range=None, domains=None, domain_expr=None, unannotated=False, evalue=None, databases=None)
Yield records with their ends trimmed.
kb_range: (left, right) kb to trim from each end, or None. cds_range: (left, right) number of CDSs to trim from each end, or None. domains, domain_expr, unannotated: also trim CDSs from each end while they have one of these domains, match this expression, or (with unannotated) have no domains; evalue and databases limit which domain annotations count. Circular contigs are linearized first. A record trimmed to nothing is dropped.
Convert between formats
domainator.genbank_to_fasta.genbank_to_fasta(input, *, fasta_out)
Write the records of the GenBank files input to the FASTA file fasta_out (- for standard output).
domainator.structure_to_genbank.structure_to_genbank(inputs, *, aligner, db_out=None, store_3di=False, tmp_dir=None)
Yield one protein SeqRecord per structure chain.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inputs
|
List[str]
|
list of -i values (structure files, globs, directories, or one database prefix). |
required |
aligner
|
StructureAligner
|
a structure_lib.StructureAligner. |
required |
db_out
|
Optional[str]
|
if not None, build the database at this prefix and leave it in place. |
None
|
store_3di
|
bool
|
store each chain's 3Di string in a /threedi qualifier. |
False
|
tmp_dir
|
Optional[str]
|
parent directory for the temporary database, when db_out is None. |
None
|
Yields:
| Type | Description |
|---|---|
SeqRecord
|
SeqRecord objects with molecule_type "protein". |
Build, select, and compare HMM and CM profiles
domainator.hmmer_build.hmmer_build(file, *, alphabet=None, name=None, acc=None, desc=None, window_length=None, window_beta=None)
Build a single plan7 profile from an MSA.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file
|
Union[str, BinaryIO]
|
path to, or open binary handle on, an MSA in any format hmmbuild accepts.
A path's suffix can decide the format ( |
required |
alphabet
|
Optional[Alphabet]
|
alphabet of the MSA. If None, pyhmmer infers it from the residues, which can guess wrong on short or ambiguous alignments. |
None
|
name
|
Optional[str]
|
value for the profile's NAME field. |
None
|
acc
|
Optional[str]
|
value for the profile's ACC field. |
None
|
desc
|
Optional[str]
|
value for the profile's DESC field. |
None
|
window_length
|
Optional[int]
|
max expected hit length (the MAXL field), used by nhmmer for windowed E-values. Nucleotide alphabets only. Takes precedence over window_beta, so only one of the two may be given. |
None
|
window_beta
|
Optional[float]
|
tail mass from which to derive the window length. Nucleotide alphabets only. |
None
|
Returns:
| Type | Description |
|---|---|
HMM
|
the built HMM. |
domainator.infernal_build.infernal_build(file, *, name, acc=None, desc=None, input_format=None, cmbuild_path=None, noss=False, hand=False, symfrac=None)
Build a covariance model with Infernal's cmbuild and return it as text.
file: an alignment (Stockholm, or FASTA with noss), as a path or a binary handle.
name, acc, desc: the model's NAME, ACC and DESC.
input_format: cmbuild's --informat, if it can't tell the format itself.
noss, hand, symfrac: cmbuild's --noss, --hand and --symfrac.
Raises ExternalToolError if cmbuild is missing or fails.
domainator.hmmer_select.hmmer_select(hmm_path=None, *, regex=None, exact=None, contains=None, hmm_iterator=None, search_name=True, search_description=True, search_accession=True, case_sensitive=False, match_all=False)
Yield the profiles of an HMM file whose name, description or accession matches.
A profile is selected if any of the regex, exact and contains strings matches
any of the fields searched.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
hmm_path
|
Optional[str]
|
the HMM file to read. |
None
|
regex
|
Optional[List[str]]
|
regular expressions to search the fields for. |
None
|
exact
|
Optional[List[str]]
|
strings that a field must equal. |
None
|
contains
|
Optional[List[str]]
|
strings that a field must contain. |
None
|
hmm_iterator
|
Optional[Iterable[HMM]]
|
pyhmmer HMMs to select from, instead of |
None
|
search_name
|
bool
|
search the profiles' NAME field. |
True
|
search_description
|
bool
|
search the profiles' DESC field. |
True
|
search_accession
|
bool
|
search the profiles' ACC field. |
True
|
case_sensitive
|
bool
|
match case. |
False
|
match_all
|
bool
|
select every profile, whatever the text criteria. The command uses it for
|
False
|
Yields:
| Type | Description |
|---|---|
HMM
|
The selected pyhmmer HMMs. |
domainator.hmmer_search.hmmer_search(input_files, *, references, hmm_out_handle, min_score=None, max_hits=None, cpu=0, max_output_bytes=None, output_description='hmmer_search HMM output')
Write the profiles of input_files that score at least min_score against any of the
references profiles to hmm_out_handle; with max_hits, only that many, the best-scoring first.
The input and reference profiles must share an alphabet. min_score None means the default
for the alphabet (see hmmer_compare). cpu 0 means one worker per physical core. Raises
OutputSizeLimitExceeded past max_output_bytes.
domainator.hmmer_compare.hmmer_compare(query_files, *, references, tsv_out_handle, min_score=0, max_hits_per_query=None, cpu=0, alignments_out_handle=None, max_output_bytes=None, output_description='hmmer_compare output')
For each hit, write a TSV row (query, reference, score) to tsv_out_handle, and the same row followed by the alignment and a blank line to alignments_out_handle. Either handle can be None.
Hits score at least min_score; with max_hits_per_query, only that many per query are
kept. cpu 0 means one worker per physical core.
Compute similarity and distance matrices
domainator.seq_dist.seq_dist(input_path, *, input_format=None, reference=None, reference_format=None, max_hits_per_query=None, algorithm='diamond_us', mode='score', cpu=0, threshold=0, progress=False, mst=False, knn=None, diamond_args=None, symmetrize='none', diamond_path=None)
Compare every sequence or profile in input_path with every one in reference.
Returns a DataMatrix of mode values with a row per input and a column per reference;
pass the same path as input_path and reference for a self-comparison. The options
mean the same as the domainator seq_dist options with the same names; diamond_args
is the parsed form of --diamond_args (see :func:parse_diamond_args). A format of None
is read from the file name; a reference of None compares input_path with itself; cpu
0 means one thread per physical core.
domainator.kmer_dist.kmer_dist(input_path, *, input_format=None, reference=None, reference_format=None, max_hits_per_query=None, metric='max_containment', mode='score', cpu=0, threshold=0, alphabet='protein', ksize=None, nucleotide_kmers=False, gene_call=None, progress=False, mst=False, knn=None)
Compare the k-mer content of every input sequence with every reference sequence.
Returns a DataMatrix of mode values with a row per input and a column per reference;
pass the same path as input_path and reference for a self-comparison. The options
mean the same as the domainator kmer_dist options with the same names. A format of None
is read from the file name; a reference of None compares input_path with itself; cpu
0 means one thread per physical core.
domainator.structure_dist.structure_dist(input_values, *, references, aligner, mode='score', evalue=0.001, max_hits_per_query=None, threshold=0.0, symmetrize='none', mst=False, knn=None, alignment_type=None, max_seqs=None, tmp_dir=None, db_out=None, hits_tsv_out=None, progress=False)
Align every input structure against every reference structure.
Returns a DataMatrix of mode values with a row per input and a column per reference.
references of None compares the input against itself. db_out and hits_tsv_out
keep the aligner database and the raw hits table.
domainator.compare_contigs.compare_contigs(records, *, metrics, max_hits_per_query=None, threshold=0.0, progress=False, cpu=0, mst=False, knn=None)
Score every pair of contigs by their domain content.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs (SeqRecords). Of records with the same id, only the first is scored. |
required |
metrics
|
Iterable[ContigMetric]
|
JaccardIndex and/or AdjacencyIndex objects; their weighted scores are summed. |
required |
max_hits_per_query
|
Optional[int]
|
keep this many of the top combined scores per contig (None: all). |
None
|
threshold
|
float
|
round combined scores less than or equal to this down to zero. |
0.0
|
progress
|
bool
|
show progress bars. |
False
|
cpu
|
int
|
number of workers (0: one per physical core). |
0
|
mst
|
bool
|
keep the maximum spanning tree edges, computed while streaming (see
|
False
|
knn
|
Optional[int]
|
keep each row's |
None
|
Returns:
| Type | Description |
|---|---|
DataMatrix
|
A square DataMatrix of "score" values, with a row and column per unique contig id. |
domainator.transform_matrix.transform_matrix(array, *, mode=PASS_THROUGH_MODE, row_lengths=None, col_lengths=None)
Return array with its scores converted to the values of mode.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
array
|
csr_array
|
a matrix of scores. |
required |
mode
|
str
|
"none" or "score" (unchanged), "bool" (1 for every hit), "norm_score", "row_norm_score",
"score_dist", "efi_score" or "efi_score_dist"; see |
PASS_THROUGH_MODE
|
row_lengths
|
Optional[ndarray]
|
the lengths of the row sequences, which the efi modes need. |
None
|
col_lengths
|
Optional[ndarray]
|
the lengths of the column sequences, which the efi modes need. |
None
|
Returns:
| Type | Description |
|---|---|
Union[csr_array, ndarray]
|
The transformed matrix: a dense array for the distance modes (score_dist, |
Union[csr_array, ndarray]
|
efi_score_dist), where pairs without a score are at the maximum distance, else sparse. |
Build, explore, and split similarity networks, trees, and projections
domainator.build_ssn.build_ssn(matrix, *, threshold=0, metadata=None, color_by=None, color_table=None, xgmml_out=None, cluster=False, cluster_tsv_out=None, no_cluster_header=False, color_table_out=None, mst=False, knn=None, subset_labels=None, max_output_bytes=None, max_color_groups=None, palette_offset=0, mark_mst=False, html_out=None, dvb_out=None, merge_impact_metric=MERGE_IMPACT_MIN_CHILD, label_by=None, categorical=None, name=None)
Write a sequence similarity network built from matrix.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
matrix
|
DataMatrix
|
symmetric matrix of edge values |
required |
threshold
|
float
|
ignore scores lower than or equal to this. Defaults to 0. |
0
|
metadata
|
List[Union[str, PathLike]]
|
tab-separated text files containing node annotations. Defaults to None. |
None
|
color_by
|
str
|
color the nodes based on their value in this metadata field. Set to SSN_cluster to color by cluster. Defaults to None. |
None
|
color_table
|
Dict[str, str]
|
maps |
None
|
xgmml_out
|
Union[str, PathLike]
|
write a Cytoscape XGMML file to this path. Defaults to None. |
None
|
cluster
|
bool
|
If True, then add new annotations with connected cluster IDs. Defaults to False. |
False
|
cluster_tsv_out
|
Union[str, PathLike]
|
write a metadata tab-separated file with the cluster ids to this path. Defaults to None. |
None
|
no_cluster_header
|
bool
|
If True, then the cluster tsv file will not have a header. Defaults to False. |
False
|
color_table_out
|
str
|
write a color table to this path. Defaults to None. |
None
|
mst
|
bool
|
If True, emit only maximum spanning tree edges, which preserves the input's connected components. Defaults to False. |
False
|
knn
|
int
|
If set, emit OR-symmetric k-nearest-neighbor edges. Combined with mst=True the result is the union of the two; on its own the emitted graph need not preserve the input's connected components. Defaults to None. |
None
|
subset_labels
|
set
|
keep only rows and columns with labels in this set. Defaults to None. |
None
|
max_output_bytes
|
int
|
raise OutputSizeLimitExceeded before writing an XGMML, HTML or dvb file larger than this. Defaults to None, meaning no limit. |
None
|
max_color_groups
|
int
|
if set, only this many of the largest color_by groups get distinct colors, the rest share a neutral gray. Defaults to None, meaning every group gets a color. |
None
|
palette_offset
|
int
|
skip this many colors at the start of the palette when assigning colors to color_by groups. Ignored when color_table is given. Defaults to 0. |
0
|
mark_mst
|
bool
|
If True, give every edge a boolean MST property saying whether it belongs to the maximum spanning tree. Defaults to False. |
False
|
html_out
|
Union[str, PathLike]
|
write the Domainator Viewer, opening this network, to this HTML file. Defaults to None. |
None
|
dvb_out
|
Union[str, PathLike]
|
write the viewer bundle (gzip-compressed JSON, .dvb) to this file. Defaults to None. |
None
|
merge_impact_metric
|
str
|
how the viewer's split chart measures a merge (MERGE_IMPACT_CHOICES). Defaults to "min_child". |
MERGE_IMPACT_MIN_CHILD
|
label_by
|
str
|
the metadata column the viewer labels nodes by at first. Defaults to None. |
None
|
categorical
|
List[str]
|
numeric metadata columns the viewer colors as categories, not a gradient. Defaults to None. |
None
|
name
|
str
|
the network's name, shown in the viewer. Defaults to the html (or dvb) output's file name. |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
if the input does not have symmetric row and column labels |
domainator.ssn_navigator.ssn_navigator(bundle, *, query='overview', threshold=None, cluster=None, node=None, min_size=1, top_n=50, members=False, max_merge_events=DEFAULT_MAX_MERGE_EVENTS)
Answer a single navigation query against a bundle with a network view, as ssn_bundle.load_bundle returns it.
Returns a JSON-serializable dict. See the module docstring for modes.
domainator.split_by_cluster.split_by_cluster(input_path, *, matrix, output_dir, threshold=0, min_cluster_size=1, write_small_clusters=False, pad_on_search_hits=False)
Write one file per cluster of matrix into output_dir, holding the records (or
profiles) of input_path that are in that cluster.
Clusters are the connected components of the matrix's entries above threshold. Clusters
smaller than min_cluster_size are skipped, or written together into one file with
write_small_clusters. Returns {"clusters": {cluster number: path}, "small_clusters": path or None}.
domainator.build_tree.build_tree(data_matrix=None, *, newick=None, algorithm='upgma', distance='auto', metadata=None, subset_labels=None, msa=None, color_by=None, color_table=None, color_table_out=None, palette_offset=0, max_color_groups=None, label_by=None, categorical=None, name=None, newick_out=None, xgmml_out=None, html_out=None, dvb_out=None, max_output_gb=DEFAULT_MAX_OUTPUT_GB, quiet=False)
Build a tree from a distance or similarity matrix, or read one from a Newick file, and write it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_matrix
|
Optional[DataMatrix]
|
a square distance or similarity matrix. Exactly one of data_matrix and newick. |
None
|
newick
|
Optional[Union[str, PathLike]]
|
read the tree from this Newick file instead of building one. |
None
|
algorithm
|
str
|
the algorithm to build the tree with. Only "upgma". |
'upgma'
|
distance
|
str
|
how to get distances from the matrix values, see condensed_distances. |
'auto'
|
metadata
|
Optional[List[str]]
|
tab-separated tables of leaf metadata, keyed by their first column. |
None
|
subset_labels
|
Optional[Set[str]]
|
only include these leaves (all of them if None). |
None
|
msa
|
Optional[Union[str, PathLike]]
|
an aligned FASTA file of the leaves' sequences, shown beside the tree in the viewer. |
None
|
color_by
|
Optional[str]
|
color the leaves in the xgmml and, at first, the viewer by this metadata column. |
None
|
color_table
|
Optional[Dict[str, str]]
|
maps color_by values to colors. None: assign colors automatically. |
None
|
color_table_out
|
Optional[str]
|
write the color table, with any colors assigned, to this path: two tab-separated columns (annotation, hex color) and no header. |
None
|
palette_offset
|
int
|
skip this many colors at the start of the palette when assigning colors to color_by values. Ignored when color_table is given. |
0
|
max_color_groups
|
Optional[int]
|
give distinct colors only to this many of the largest color_by groups. |
None
|
label_by
|
Optional[str]
|
the metadata column the viewer labels leaves by at first. |
None
|
categorical
|
Optional[List[str]]
|
numeric metadata columns the viewer colors as categories, not a gradient. |
None
|
name
|
Optional[str]
|
the tree's name, shown in the viewer. Defaults to the html or dvb output's stem. |
None
|
newick_out
|
Optional[str]
|
write the tree to this Newick file. |
None
|
xgmml_out
|
Optional[str]
|
write the tree to this Cytoscape xgmml file, laid out as a rectangular tree. |
None
|
html_out
|
Optional[str]
|
write the Domainator Viewer, opening this tree, to this HTML file. |
None
|
dvb_out
|
Optional[str]
|
write the viewer bundle (gzip-compressed JSON, .dvb) to this file. |
None
|
max_output_gb
|
float
|
refuse to write an html or dvb output larger than this (0: no limit). |
DEFAULT_MAX_OUTPUT_GB
|
quiet
|
bool
|
do not warn about leaf names changed for Newick. |
False
|
Returns:
| Type | Description |
|---|---|
Tree
|
The tree. |
domainator.build_projection.build_projection(data_matrix, *, algorithm='pca', metadata=None, color_by=None, html_out=None, dvb_out=None, coords_out=None, xgmml_out=None, scale=1000, three_d=False, algorithm_parameters=None, color_table=None, color_table_out=None, palette_offset=0, label_by=None, categorical=None, name=None)
Project the rows of data_matrix into 2D or 3D and write the projection.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_matrix
|
DataMatrix
|
rows are the points (typically sequences), columns their features (typically scores against other sequences or profiles). |
required |
algorithm
|
str
|
"pca", "tsne", "umap", or "coords" (the matrix already holds the coordinates). |
'pca'
|
metadata
|
Optional[List[str]]
|
tab-separated metadata files, joined to the points by their first column. |
None
|
color_by
|
Optional[str]
|
color the points by this metadata column. |
None
|
html_out
|
Optional[str]
|
write the Domainator Viewer, showing the projection, to this HTML file. |
None
|
dvb_out
|
Optional[str]
|
write the viewer bundle (gzip-compressed JSON, .dvb) to this file. |
None
|
coords_out
|
Optional[str]
|
write the coordinates to this tab-separated file. |
None
|
xgmml_out
|
Optional[str]
|
write the projection to this Cytoscape XGMML file. |
None
|
scale
|
Optional[float]
|
scale the coordinates so that the widest axis spans this much. |
1000
|
three_d
|
bool
|
project into 3D instead of 2D. |
False
|
algorithm_parameters
|
Optional[Dict[str, Any]]
|
keyword arguments for the projection algorithm. For "pca" on a sparse matrix, which is projected with scikit-learn's TruncatedSVD instead of PCA, only the keywords the two share apply. |
None
|
color_table
|
Optional[Dict[str, str]]
|
maps |
None
|
color_table_out
|
Optional[str]
|
write the color table, with any colors assigned, to this path: two tab-separated columns (annotation, hex color) and no header. |
None
|
palette_offset
|
int
|
skip this many colors at the start of the palette when assigning colors to color_by values. Ignored when color_table is given. |
0
|
label_by
|
Optional[str]
|
the metadata column the viewer labels points by at first. |
None
|
categorical
|
Optional[List[str]]
|
numeric metadata columns the viewer colors as categories, not a gradient. |
None
|
name
|
Optional[str]
|
the projection's name, shown in the viewer. Defaults to the html or dvb output's stem. |
None
|
domainator.viewer.viewer(input=None, *, html_out, name=None)
Write the Domainator Viewer page.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
Optional[Union[str, PathLike]]
|
a viewer bundle (.dvb, or an older network bundle, .dsnv) for the page to open on load. None: the page opens bundles chosen in it. |
None
|
html_out
|
Union[str, PathLike]
|
write the page to this HTML file. |
required |
name
|
Optional[str]
|
the page's title. Defaults to the bundle's name. |
None
|
Tables and summaries
domainator.enum_report.enum_report(records, *, by='contig', analyses, tsv_out_handle=None, html_out_handle=None, column_names=None, html_max_height=None, ncbi_taxonomy=None, databases=None, json_out_handle=None, partial='include')
Write a table with one row per contig, CDS or domain of records.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the contigs or proteins (SeqRecords). |
required |
by
|
str
|
what each row describes: "contig", "cds" or "domain". For proteins, contig and cds are the same. |
'contig'
|
analyses
|
List[Analysis]
|
the columns to report (from report_columns.parse_columns). |
required |
tsv_out_handle
|
Optional[TextIO]
|
an open text handle for the TSV table, or None. |
None
|
html_out_handle
|
Optional[TextIO]
|
an open text handle for the HTML table, or None. |
None
|
column_names
|
Optional[List[str]]
|
use these column names instead of the default ones. |
None
|
html_max_height
|
Optional[int]
|
the largest height of the HTML table, in pixels. |
None
|
ncbi_taxonomy
|
Optional[NCBITaxonomy]
|
an NCBITaxonomy, needed by the taxonomy columns. |
None
|
databases
|
Optional[Set[str]]
|
report only domains from these databases. None: all. |
None
|
json_out_handle
|
Optional[TextIO]
|
an open text handle for NDJSON (one object per row), or None. |
None
|
partial
|
str
|
which rows to report, by fragment status: "include" (all, the default), "exclude" (skip fragments), or "only" (only fragments). See process_record. |
'include'
|
domainator.hmmer_report.hmmer_report(records, *, analyses, tsv_out_handle=None, html_out_handle=None, column_names=None, html_max_height=None, json_out_handle=None)
Write a table with one row per profile of records.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[Tuple[str, Iterable[HMM]]]
|
(source name, iterator of pyhmmer HMMs) pairs, one per file, as read_hmms_to_iterators returns. |
required |
analyses
|
List[Analysis]
|
the columns to report (from report_columns.parse_columns). |
required |
tsv_out_handle
|
Optional[TextIO]
|
an open text handle for the TSV table, or None. |
None
|
html_out_handle
|
Optional[TextIO]
|
an open text handle for the HTML table, or None. |
None
|
column_names
|
Optional[List[str]]
|
use these column names instead of the default ones. |
None
|
html_max_height
|
Optional[int]
|
the largest height of the HTML table, in pixels. |
None
|
json_out_handle
|
Optional[TextIO]
|
an open text handle for NDJSON (one object per profile), or None. |
None
|
domainator.summary_report.summary_report(records, *, text_out_handle=None, html_out_handle=None, domains=None, co_occurrence_fraction_out=None, co_occurrence_count_out=None, taxonomy=False, ncbi_taxonomy=None, domains_tsv_out=None, databases=None, json_out_handle=None, partial='include')
Write a summary of records: contig and CDS counts, contig lengths and a table of the
domains found, to any of the text, HTML and JSON handles (each can be None).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[SeqRecord]
|
the annotated contigs or proteins (SeqRecords). |
required |
text_out_handle
|
Optional[TextIO]
|
an open text handle for the text report. |
None
|
html_out_handle
|
Optional[TextIO]
|
an open text handle for the HTML report. |
None
|
json_out_handle
|
Optional[TextIO]
|
an open text handle for the JSON report. |
None
|
domains
|
Optional[Set[str]]
|
focus domains: the report gives how often each other domain occurs with them, and how far apart they are. Co-occurrence rows are ordered alphabetically by focus domain. |
None
|
co_occurrence_fraction_out
|
Optional[str]
|
a path for the domain-by-domain co-occurrence matrix, as fractions of contigs. |
None
|
co_occurrence_count_out
|
Optional[str]
|
a path for the co-occurrence matrix as counts of contigs. |
None
|
taxonomy
|
bool
|
also summarize the records' taxonomy, using |
False
|
domains_tsv_out
|
Optional[str]
|
a path for the domain table as TSV. |
None
|
databases
|
Optional[Set[str]]
|
count only domains from these databases. |
None
|
partial
|
str
|
which CDSs/proteins to count, by fragment status: "include" (all, the default), "exclude" (skip fragments), or "only" (only fragments). Protein records that are filtered out are skipped entirely. On nucleotide contigs, filtered CDSs and the domains on them are not counted, but the contig is. See partial.get_fragment_status. |
'include'
|
domainator.matrix_report.matrix_report(matrix, *, text_out_handle=None, html_out_handle=None, knn_projection=False, merge_impact_metric=MERGE_IMPACT_MIN_CHILD, stage_timings=None, progress_callback=None, max_merge_events=DEFAULT_MAX_MERGE_EVENTS, json_out_handle=None)
Write a report on matrix, a similarity matrix, to any of the text, HTML and JSON
handles (each can be None): its size and value distribution, and for a symmetric matrix,
how its clusters merge as the threshold falls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
knn_projection
|
bool
|
also project the edge counts of |
False
|
merge_impact_metric
|
str
|
how split events are measured, "min_child" or "product". |
MERGE_IMPACT_MIN_CHILD
|
max_merge_events
|
int
|
how many of the strongest merge events to embed (0: all). |
DEFAULT_MAX_MERGE_EVENTS
|
stage_timings
|
Optional[List[tuple]]
|
a list to append each stage's (stage, seconds, metrics) timing to. |
None
|
progress_callback
|
Optional[Callable[[str], None]]
|
called with the name of each stage as it starts. |
None
|
Figures
domainator.plot_contigs.plot_contigs(records, *, html_out, metadata=None, color_by=None, color_table=None, palette_offset=0, height=2400, width=1200, types=None, feature_height=50, arrow_width=20, contig_thickness=20, contig_label_font='sans-serif', contig_label_px=32, feature_label_font='sans-serif', feature_font_size=26, details_height=100, details_width=None)
Write an interactive HTML plot of records to html_out (a path or a text handle).
The options mean the same as the domainator plot_contigs options with the same names;
types is the feature types to draw, and palette_offset is ignored when
color_table is given. A contig with no color_by value is drawn in OTHER_COLOR, and
one whose value color_table does not list in DEFAULT_CONTIG_COLOR.
domainator.color_table_to_legend.color_table_to_legend(table, *, svg_out, title='Legend')
Write an SVG legend of table (annotation -> hex color) to svg_out, a path (- for stdout).
Download and prepare databases
domainator.db_download.db_download(ncbi_taxonomy, *, output, include_taxids=None, exclude_taxids=None, db, gene_call=None, max_records=None, ncbi_summary=None, before=None, after=None, cpu=0, success_log_out=None, exclude_accessions=None, download_backend='datasets', download_workers=3, user_agent=None, datasets_workdir=None, datasets_include='gbff', datasets_max_workers=10, ncbi_api_key=None, datasets_path=None, datasets_gzip=False, taxonomy_expr=None, partial='include', uniprot_dir=None)
Download a database (db: a UniProt or NCBI genome database) and write it to output.
UniProt databases are written as FASTA, or GenBank for the _gb ones; NCBI genome databases as
GenBank. include_taxids, exclude_taxids and taxonomy_expr filter by taxonomy (with
ncbi_taxonomy, an NCBITaxonomy); max_records stops after that many records; before and
after filter NCBI genomes by release date; success_log_out lists the accessions written;
uniprot_dir reads the UniProt release files from a local directory instead of downloading
them. Records are appended to output. The other options are those of
domainator db_download.
domainator.format_db.format_db(inputs, *, shards=None, compress=False, index=False, output_dir=None, name=None, cpu=0, level=6, force=False, log_out_handle=sys.stderr)
Shard, compress and/or index domain_search databases, and return the output paths.
At least one of shards, compress and index must be requested.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inputs
|
List[str]
|
GenBank or FASTA database files (plain, gzip or BGZF). |
required |
shards
|
Optional[int]
|
split each input into this many shards, balanced by record count. |
None
|
compress
|
bool
|
write BGZF (block-gzip) output. |
False
|
index
|
bool
|
write a .didx offset index for each output file. |
False
|
output_dir
|
Optional[str]
|
write the outputs here. None: next to each input. |
None
|
name
|
Optional[str]
|
a new base name for the output database (its format and compression extensions are kept). Requires a single input. |
None
|
cpu
|
int
|
number of parallel workers (also the BGZF compressor's threads). 0: one per physical core. |
0
|
level
|
int
|
the BGZF compression level (1-12). |
6
|
force
|
bool
|
overwrite existing outputs and rebuild indexes instead of skipping them. |
False
|
log_out_handle
|
TextIO
|
where to write the log. |
stderr
|
Returns:
| Type | Description |
|---|---|
List[str]
|
The output paths written (or found up to date). |