#acl All:read This page is now superseded by http://bridgedb.org/wiki/ComparisonMatrix ------ Database Identifier Mapping use cases Links between database identifiers are required for five specific use-cases. Most of the database identifiers are for molecules (proteins, small molecules), but identifiers for complexes, interactions, pathways, molecular states, etc. are also required, but are less important in the short to mid term (e.g. next 3-5 years). Use cases: 1. '''Unification''' during dataset merging: During a merge operation e.g. of two protein-protein interaction datasets from independently created databases, it is vital to recognize that two protein objects, one from each data source, represent the same protein molecule, even if the protein objects don’t share any database accession numbers. Unification requires knowledge of record type e.g. you cannot reliably use a gene ID to unify proteins (mostly because splice variants exist). 1. '''Link out''' to related references: When presenting information about a protein to a user on a web page, it is useful to display links to related information about the protein, such as further information about the protein sequence and sequence feature annotations (e.g. in !UniProt), Gene Ontology annotations, domains annotations (!InterPro), etc. 1. '''Identifier translation''': Some analysis methods require specific translations from one set of identifiers to another. For instance, our ‘activity centers’ analysis requires translation from protein or gene identifiers in a pathway database to Affymetrix probe set identifiers or other gene expression array platform identifiers. 1. '''Searching''' for a favorite gene name: Preferred gene names used for querying a pathway database should return all genes/proteins with that name, if they exist in the database. Unlike database accession numbers, gene names are not guaranteed unique, thus cannot reliably be used for the other use cases. 1. Special case of identifier translation between species via orthology links. Links are available from many sources, but not every source addresses each use case (and none address all use cases). All services that allow all data to be downloaded can conceivably be used for all use cases with the help of a separate software system that can store different link types (e.g. unification links, link out links), although this also requires recognition of record type. __'''Mapping services'''__ CRONOS: the cross-reference navigation server http://bioinformatics.oxfordjournals.org/cgi/content/abstract/25/1/141?etoc !AliasServer<
> http://cbi.labri.fr/outils/alias/ <
> Tool for identifier translation using CRC64 hash of the protein sequence as a primary key. Provides unification, linkout and translation services for a handful (~35) species for proteins only. Supports use cases 2, 3.<
> Freely available for download. Regularly updated. MD Anderson !GeneLink<
> http://bioinformatics.mdanderson.org/GeneLink.html <
> ID translation and search service for human IDs (10 ID types). Supports use case 2, 3. !EnsMart<
> http://www.ensembl.org/Multi/martview <
> ID translation services for Ensembl genomes. Supports use case 2. !MatchMiner<
> http://discover.nci.nih.gov/matchminer/ <
> ID translation service for mouse and human. Supports use case 2, 3. Ariadne Genomics ID Mapping Service<
> http://www.ariadnegenomics.com/services/idmap.html <
> Tool for identifier translation. Supports 7 species and maps between 12 different ID types mainly for proteins and genes.<
> Commercial service, not available for download<
> Supports use case 3 !GeneLynx<
> http://www.genelynx.org/ <
> Provides linkout services for human, mouse and rat. Supports use case 2. !NetAffx<
> http://www.affymetrix.com/products/software/specific/netaffx.affx <
> Provides ID translation services for Affy probe set IDs. Supports use case 3. PIR ID mapping<
> http://pir.georgetown.edu/pirwww/search/idmapping.shtml <
> Provides ID translation sservices for many different ID types. Supports use case 3<
> You can bypass the webpage and do this mapping programmatically via an API and URL, if interested in details of these functions or to request additional mappings contact us for details at pirmail . AT .georgetown.edu A table with 15 of the most requested protein id mappings can be downloaded by ftp see: ftp://ftp.pir.georgetown.edu/databases/iproclass/iproclass.tb.readme for a list of the current IDs, they are mostly protein centric IDs but EntrezGene , GO, Unigene and OMIM are included. The table is found at ftp://ftp.pir.georgetown.edu/databases/iproclass/iproclass.tb.gz (it is large, 124MB) The caBIG !GeneConnect project will provide APIs and webpages to map between the following ID, however some mappings will be limited to Human, Mouse and Rat. * Ensembl Gene ID, Ensembl Transcript ID, Ensembl Protein ID, Entrez Gene ID, !UniGene ID, !GenBank mRNA Accession Number, !GenBank Protein Accession Number, !RefSeq mRNA Accession Number, !RefSeq Protein Accession Number, UniProtKB Primary Accession Number * https://cabig.nci.nih.gov/tools/GeneConnect/ Synergizer project from Fritz Roth's lab: http://llama.med.harvard.edu/cgi/synergizer/translate * Supports use case 3 Onto-Translate at http://vortex.cs.wayne.edu/Projects.html#Onto-Translate [[http://www.biomedcentral.com/1471-2105/8/9|IDconverter and IDClight]] __'''Databases'''__ International Protein Index<
> http://www.ebi.ac.uk/IPI/IPIhelp.html <
> A cross reference database for proteins in higher eukaryotic organisms (5 species). Provides protein and gene cross references. Supports use case 1. Entrez Gene<
> Provides detailed information on genes from multiple organisms including gene aliases and links to NCBI related resources. Supports use case 4 (and 2 to some degree). !UniProt (!SwissProt, PIR, TrEMBL) provides some information on links to related resources and protein names. http://www.genmapp.org has identifier mapping databases for many model organisms. http://ensembl.org provides species specific identifier mappings and these are accessible via the !BioMart web service. (Originally from a document written for cPath (MSKCC Pathway Database) by Gary Bader, Wednesday March 9, 2005) ---- General comments: Nicolas Le Novere - May.10.2006 on biopax discuss mailing list {{{ Related to the issue of unique but non-resolvable URIs, I would like to introduce MIRIAM annotation framework. The annotation of models is an important part of the MIRIAM standard of curation. MIRIAM is described in: Le Novère N., Finney A., Hucka M., Bhalla U., Campagne F., Collado-Vides J., Crampin E., Halstead M., Klipp E., Mendes P., Nielsen P., Sauro H., Shapiro B., Snoep J.L., Spence H.D., Wanner B.L. (2005) Minimum Information Requested In the Annotation of biochemical Models (MIRIAM) Nature Biotechnology, 23: 1509-1515. In order to have homogenous annotation, we came up with a system using URIs and accessions to uniquely describe a data, but let the choice to the user when it comes to use this information. We are now developing a unique resource that will provide access to a unified set of URI, plus additional information, like regexp to validate the IDs, name and documentation about the data resources etc. The URIs are different from the physical locations, and a given URIs can be resolved into several alternative locations. http://www.ec-code.org/#1.1.1.1 can be transformed into: http://www.ebi.ac.uk/intenz/query?cmd=SearchEC&ec=1.1.1.1 http://www.genome.jp/dbget-bin/www_bget?ec:1.1.1.1 http://us.expasy.org/cgi-bin/nicezyme.pl?1.1.1.1 http://www.taxonomy.org/#9606 can be transformed into: http://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&id=9606 http://www.ebi.ac.uk/newt/display?search=9606 etc. We also set-up a system of deprecation, so that if several resources are merged (Soemthing that is fortunately more and more frequent), we will always retrieve the data based on the old URIs. The database (MySQL) is now ready, also not public yet (you can find snapshot of the URIs at: http://sbml.org/wiki/MIRIAM_URI_Set). As a real-life test, the new version of BioModels Database, planned to be released this month will use it to resolve all the model annotation. We started to develop webservices so that URIs can be resolved into URLs etc. }}} Gary Bader: May.10.2006 {{{ Hi Nicolas, So are you guys going to maintain a list of general "linkout" URLs? Or are you just proposing a standard way of representing these and people can maintain their own lists? Either would be great! A lot of groups are maintaining internal lists. For instance, the cPath list is stored in a SQL table and currently contains 24 entries in the form e.g. 'UniProt', 'http://www.pir.uniprot.org/cgi-bin/upEntry?id=%ID%', 'UniProt (Universal Protein Resource) is the world\'s most comprehensive catalog of information on proteins.' Note that we have %ID% as a symbol to replace with the actual ID. Occasionally a resource will require an extra string at the end of the URL (e.g. ".html") - this was the case for HPRD until they recently changed their URL scheme. Cytoscape also has a linkout plugin that reads a list of links from a file to populate a linkout menu e.g. url.yeast.BioGRID=http://www.thebiogrid.org/search.php?keywords=%ID%&searchbutton=GO&organismid=4932 url.google=http://www.google.com/search?hl=en&q=%ID%&btnG=Google+Search Also, I'm pretty sure that IntAct is maintaining a list of linkouts in addition to the OBO file of official database names made available as part of the PSI-MI controlled vocabularies. E.g. http://cvs.sourceforge.net/viewcvs.py/intact/intactCore/data/controlledVocab/CvDatabase.def?rev=1.19&view=auto (I'm not sure where the linkout list is, if it exists) And Chris Lemer and the Northbears team (aMAZE group) created an OWL ontology describing databases for the KEGG2BioPAX converter. It has around 200 databases in it (all the ones referenced by KEGG). A few example entries are below. (the full file is available from the Northbears CVS server at biopax.data/data/extra/db-alias.owl (access instructions are here http://www.northbears.org/biopax/quick_guide.shtml ) It would be great if we could create a standard format for all this information that we could use for 2 use cases: 1. Validating external references - both that the database name is recognized and that the IDs follow the correct pattern 2. Using to create hyperlinks to other resources for more information. If you guys are willing to push this, I will definitely contribute as a reviewer, by inputing actual URLs and by supporting it in cPath, Cytoscape, and to propose it for use in validating BioPAX. This would be immediately useful at least to the projects mentioned above. If the format is complex, I would say that an additional requirement is to have a simple tab delimited version of whatever the format is (e.g. if it is in OWL) to encourage people to use it. Thanks, Gary Example db-alias.owl entries: KEGG PATHWAY KEGG KEGG KEGG Database KEGG Database KEGG PATHWAY KEGG PATHWAY ----- The Institute for Genomic Research --- and the ontology is: 1 1 A Database represents a structured collection of data. It is governed by one and only one authority which is an Organization. An Organization is a governing authority over a Database. 1 A RelationshipDatabaseReference is a reference made by one Database to another which does not share the same identity but has a semantic link with the referring instance. 1 A DatabaseReference is a reference made by one instance of Database to one instance of another Database. This reference can be one of 2 types: RelationshipDatabaseReference or UnificationDatabaseReference. A Reference can have several names pointing to the same Database. 1 A UnificationDatabaseReference is a reference made by one Database to another which shares the same identity. An entity of a certain type in the referencing database, references an entity of the same type in the referenced database. The Organization that governs the Database. The Database referenced by a DatabaseReferenced. The Database referenced by the current Database. The Database embedding the current Database. The name given by the entity to itself. The name given by a Database to a referenced Database. The type of relationship of a RelationshipDatabaseReference. }}} Nicolas Le Novere: May.10.2006 {{{ We will indeed maintain a general list of stable, maintained data resources, whether databases or ontologies. The original goal was to help modellers to to generate consistent annotation and resolve existing annotation. Of course anyone will be able to propose new type of data or data resource. Each data type is very similar to the cPath list you mentioned: names (1 official plus synonyms), e.g. "GO" "Gene Ontology" URIs (1 official plus deprecated ), e.g. http://www.ec-code.org/ Locations: For each one, the URL of the resource, and the base URL used to build an hyperlink to a precise data. In addition, there is a regexp fo reach datatype, and documentation fo rthe resources and the locations. We are now developing webservices, so one can do something like: getURI("UniProt","P12345"), and get back http://www.uniprot.org/#P12345 or getURLs("http://www.uniprot.org/#P12345") and get back: http://www.ebi.uniprot.org/entry/P12345 http://us.expasy.org/uniprot/P12345 http://www.pir.uniprot.org/cgi-bin/upEntry?id=P12345 > > A lot of groups are maintaining internal lists. I see that. This is awesome. We will be able to shamefully steal all their work. Thanks fo the info. We should definitively link to all those efforts. > > It would be great if we could create a standard format for all this > > information that we could use for 2 use cases: > > 1. Validating external references - both that the database name is > > recognized and that the IDs follow the correct pattern > > 2. Using to create hyperlinks to other resources for more information. That is definitively our aim. Our current XML export is quite (too) simple, but we wanted something easy to parse. > > If you guys are willing to push this, I will definitely contribute as a > > reviewer, by inputing actual URLs and by supporting it in cPath, > > Cytoscape, and to propose it for use in validating BioPAX. This would > > be immediately useful at least to the projects mentioned above. That would be an unvaluable help indeed. I heard you are coming at the EBI at the end of June. Unfortunately I will be in Oslo those days, but you should meet Camille Laibe, currently in charge of the development of MIRIAM database and webservices. > > If the format is complex, I would say that an additional requirement is > > to have a simple tab delimited version of whatever the format is (e.g. > > if it is in OWL) to encourage people to use it. The master data is in MySQL tables. I don't think it is a huge problem to generate various formats at all. I have a feeling that our need is actually a very common one, and there is maybe room for a more ambitious project, on "data integration for Systems Biology", and maybe a grant application together with other partners ... I was asked to write a review for Nature Cell Biology with Henning Hermjakob. I could start to instill the idea to the community. }}} Henning Hermjakob: May 12.2006 {{{ As you know, we have exactly the same problem, and I'd be interested in a general solution. What we can offer so far is the PSI MI xrefs (which you know), available in OBO format from http://psidev.sourceforge.net/mi/rel25/data/psi-mi25.obo (the sourceforge CVS is still buggy at the moment) and interactively and as a web service from http://www.ebi.ac.uk/ontology-lookup/browse.do?ontName=MI&termId=MI%3A0444&termName=database%20citation }}} Nicolas Le Novere: May 12.2006 {{{ Henning, We can certainly use your XRefs to feed MIRIAM database. I think our resources (MIRIAM and cPATH) present certain advantages for a general use by the community (In particular, there is a clear difference between a datatype and its instantiation in a database. Each data-type is defined by a perennial URI, independant of the various URLs). However we can definitively cross-fertilize, not only with the content, but also the structure. For instance, I noticed that you had definitions on each entry. Quite astonishingly, we did not think about that!! (Camille, one extra-task for you). Let's keep each other posted. }}} Alan Ruttenberg - Jun.19.2006 on biopax-discuss mailing list - post related to defining a controlled vocabulary of database names, to be used for xrefs to help match identical xrefs from different sources. * http://mumble.net/~alanr/cshl/URICV.htm * http://biopaxwiki.org/cgi-bin/moin.cgi/DatabaseRegistry * http://www.geneontology.org/cgi-bin/xrefs.cgi From Nicolas Le Novere - Jun.19.2006 on biopax-discuss mailing list {{{ Le Novère N., Finney A., Hucka M., Bhalla U., Campagne F., Collado-Vides J., Crampin E., Halstead M., Klipp E., Mendes P., Nielsen P., Sauro H., Shapiro B., Snoep J.L., Spence H.D., Wanner B.L. (2005) Minimum Information Requested In the Annotation of biochemical Models (MIRIAM). Nature Biotechnology, 23: 1509-1515. http://dx.doi.org/10.1038/nbt1156 The new version of SBML has a formal way of using those UIRs: http://sbml.org/wiki/sbml-level-2-version-2.pdf See section 6. Since then, we have been developing a database to store those standard URIs and all the related information (name, synonyms, URLs, regular-expression to parse accession, associated documentation etc.) We developed WebServices to access the database. Therefore when a tool will parse an SBML file, it will be able to transform the URIs into physical URLs etc. Conversely, a tool will be able to generate the correct URI using the name of a resource and an accession. The database is not publicly available yet, but we already use it for BioModels Database purpose. Our plan is to release the whole lot by the end of August. But we could put a demo online before that. My suggestion is for BioPAX to use MIRIAM URIs. I see two great advantages to that: 1) our aim being exactly the same, it would eliminate some redundant work. 2) SBML and BioPAX constituents would be annotated using the same URIs, making the conversion so easier. }}}