To have the most up-to-date version with a small, sample dataset:
git clone https://github.com/HelloJocelynLu/t5chem.git
cd t5chem/
python setup.py install
You will find a sample dataset under data/ directory. Don't forget to extract it before using!
To install via pip, you will need to prepare your own datasets or download some datasets to run a model:
pip install t5chem
You will find a sample dataset under data/ directory. Don't forget to extract it before using!
We have some sample data (a small subset from datasets used in paper, ~2.0 M) available in data/ folder if installed from source, you can also find a copy of sample dataset here. To have a quick start, extract the data file:
tar -xjvf data/sample_data.tar.bz2
Feel free to find full datasets in Home page. You are also encouraged to try your own datasets with the following files:
The simplest data folder should contain train.source, train.target for training, and test.source, test.target for testing. Optionally, you will also need val.source, val.target for validation.
T5Chem supports directly call from comment line, and one can check its manual by:
t5chem -h # show the general help information
t5chem train -h # show help information for model training
t5chem predict -h # show help information for model prediction
T5Chem can be directly trained from scratch. But it is recommended to let it train from a pre-trained model. We have a pre-trained model available in Home page, it was trained on 97 million PubChem Molecules with BERT-like self-supervised mask-filling scheme. It needs to be loaded as --pretrain model for T5Chem and fine-tuned for any down-streaming tasks. You can also find more models, including a well-fine-tuned model that do not need to train in Home page. To extract model weights:
tar -xjvf model/simple_pretrain.tar.bz2
Let's make a quick demo on a small sample data:
t5chem train --data_dir data/sample/product/ --output_dir model/ --task_type product --pretrain models/pretrain/simple/ --num_epoch 30
It takes about 15 minutes to finish training on RTX8000 GPU. Then to test model performance on testset:
t5chem predict --data_dir data/sample/product/ --model_dir model/
That's it! Now we get a quick demo model on forward reaction prediction task. Note that we may not get a very good result (~70% top-1 accuracy) as we are only trained on a very small sample data.
Below are some required arguments:
data_dir: The path to data directory, should contain train.source, train.target for training, and test.source, test.target for testing.
output_dir: The directory to save trained model. Will generate four files: vocab.pt for vocabulary, config.json for model configuration, pytorch_model.bin for trained model weights and training_args.bin for training arguments.
task_type: Task type. Currently we are supporting product (for forward reaction prediction), reactants (for single-step retrosynthesis), reagents (for reagents prediction), regression (to predict numeric values) and classification (to predict categorical values).
model_dir: The path to a trained model. (task type, tokenization method...etc can be inferred from saved model.
One can import t5chem package and use it in python project:
import os
from transformers import T5ForConditionalGeneration
from t5chem import T5ForProperty, SimpleTokenizer
pretrained_path = "path_to_model_directory"
model = T5ForConditionalGeneration.from_pretrained(pretrain_path) # for seq2seq tasks
tokenizer = SimpleTokenizer(vocab_file=os.path.join(pretrain_path, 'vocab.pt'))
inputs = tokenizer.encode("Product:COC(=O)c1cc(COc2ccc(-c3ccccc3OC)cc2)c(C)o1.C1CCOC1>>", return_tensors='pt')
output = model.generate(input_ids=inputs, max_length=300, early_stopping=True)
tokenizer.decode(output[0], skip_special_tokens=True) # "COc1ccccc1-c1ccc(OCc2cc(C(=O)O)oc2C)cc1"
model = T5ForProperty.from_pretrained(pretrain_path) # for non-seq2seq task
inputs = tokenizer.encode("Classification:COC(=O)c1cccc(C(=O)OC)c1>CN(C)N.Cl.O>COC(=O)c1cccc(C(=O)O)c1", return_tensors='pt')
outputs = model(inputs)
print(outputs.logits.argmax()) # Class 3We have Google Colab examples available! Feel free to try it out:
Call T5Chem via CLI (command line) link
Use a pretrained model in pythonlink
Design your own project: predict molecular weights link
1. Get ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/site-packages/rdkit/DataStructs/cDataStructs.so) (See here)
Try to re-install rdkit, for example:
conda install -q -y -c conda-forge rdkit
2. Get torchtext: No argument named max_size
Try to install torchtext version earlier than 0.10.0. torchtext version 0.10.0 published some backward incompatible changes. (See here)