tor

torchscan

Useful information about PyTorch modules (FLOPs, MACs, receptive field, etc.)

Showing:

Popularity

Downloads/wk

0

GitHub Stars

76

Maintenance

Last Commit

2mos ago

Contributors

1

Package

Dependencies

1

License

MIT

Categories

Readme

Torchscan: meaningful module insights

License: Apache 2.0 Codacy Badge Build Status codecov Docs Pypi

The very useful summary method of tf.keras.Model but for PyTorch, with more useful information.

Quick Tour

Inspecting your PyTorch architecture

Similarly to the torchsummary implementation, torchscan brings useful module information into readable format. For nested complex architectures, you can use a maximum depth of display as follows:

from torchvision.models import densenet121
from torchscan import summary

model = densenet121().eval().cuda()
summary(model, (3, 224, 224), max_depth=2)

which would yield

__________________________________________________________________________________________
Layer                        Type                  Output Shape              Param #        
==========================================================================================
densenet                     DenseNet              (-1, 1000)                0              
├─features                   Sequential            (-1, 1024, 7, 7)          0              
|    └─conv0                 Conv2d                (-1, 64, 112, 112)        9,408          
|    └─norm0                 BatchNorm2d           (-1, 64, 112, 112)        257            
|    └─relu0                 ReLU                  (-1, 64, 112, 112)        0              
|    └─pool0                 MaxPool2d             (-1, 64, 56, 56)          0              
|    └─denseblock1           _DenseBlock           (-1, 256, 56, 56)         338,316        
|    └─transition1           _Transition           (-1, 128, 28, 28)         33,793         
|    └─denseblock2           _DenseBlock           (-1, 512, 28, 28)         930,072        
|    └─transition2           _Transition           (-1, 256, 14, 14)         133,121        
|    └─denseblock3           _DenseBlock           (-1, 1024, 14, 14)        2,873,904      
|    └─transition3           _Transition           (-1, 512, 7, 7)           528,385        
|    └─denseblock4           _DenseBlock           (-1, 1024, 7, 7)          2,186,272      
|    └─norm5                 BatchNorm2d           (-1, 1024, 7, 7)          4,097          
├─classifier                 Linear                (-1, 1000)                1,025,000      
==========================================================================================
Trainable params: 7,978,856
Non-trainable params: 0
Total params: 7,978,856
------------------------------------------------------------------------------------------
Model size (params + buffers): 30.76 Mb
Framework & CUDA overhead: 423.57 Mb
Total RAM usage: 454.32 Mb
------------------------------------------------------------------------------------------
Floating Point Operations on forward: 5.74 GFLOPs
Multiply-Accumulations on forward: 2.87 GMACs
Direct memory accesses on forward: 2.90 GDMAs
__________________________________________________________________________________________

Results are aggregated to the selected depth for improved readability.

For reference, here are explanations of a few acronyms:

  • FLOPs: floating-point operations (not to be confused with FLOPS which is FLOPs per second)
  • MACs: mutiply-accumulate operations (cf. wikipedia)
  • DMAs: direct memory accesses (many argue that it is more relevant than FLOPs or MACs to compare model inference speeds cf. wikipedia)

Additionally, for highway nets (models without multiple branches / skip connections), torchscan supports receptive field estimation.

from torchvision.models import vgg16
from torchscan import summary

model = vgg16().eval().cuda()
summary(model, (3, 224, 224), receptive_field=True, max_depth=0)

which will add the layer's receptive field (relatively to the last convolutional layer) to the summary.

Setup

Python 3.6 (or higher) and pip/conda are required to install Torchscan.

Stable release

You can install the last stable release of the package using pypi as follows:

pip install torchscan

or using conda:

conda install -c frgfm torchscan

Developer installation

Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:

git clone https://github.com/frgfm/torch-scan.git
pip install -e torch-scan/.

Benchmark

Below are the results for classification models supported by torchvision for a single image with 3 color channels of size 224x224 (apart from inception_v3 which uses 299x299).

ModelParams (M)FLOPs (G)MACs (G)DMAs (G)RF
alexnet61.11.430.710.72195
googlenet6.623.011.511.53--
vgg11132.8615.237.617.64150
vgg11_bn132.8715.267.637.66150
vgg13133.0522.6311.3111.35156
vgg13_bn133.0522.6811.3311.37156
vgg16138.3630.9615.4715.52212
vgg16_bn138.3731.0115.515.55212
vgg19143.6739.2819.6319.69268
vgg19_bn143.6839.3419.6619.72268
resnet1811.693.641.821.84--
resnet3421.87.343.673.7--
resnet5025.568.214.114.15--
resnet10144.5515.667.837.9--
resnet15260.1923.111.5611.65--
inception_v327.1611.455.735.76--
squeezenet1_01.251.640.820.83--
squeezenet1_11.240.70.350.36--
wide_resnet50_268.8822.8411.4311.51--
wide_resnet101_2126.8945.5822.822.95--
densenet1217.985.742.872.9--
densenet16128.6815.597.797.86--
densenet16914.156.813.43.44--
densenet20120.018.74.344.39--
resnext50_32x4d25.038.514.264.3--
resnext101_32x8d88.7932.9316.4816.61--
mobilenet_v23.50.630.310.32--
shufflenet_v2_x0_51.370.090.040.05--
shufflenet_v2_x1_02.280.30.150.15--
shufflenet_v2_x1_53.50.60.30.31--
shufflenet_v2_x2_07.391.180.590.6--
mnasnet0_52.220.220.110.12--
mnasnet0_753.170.450.230.24--
mnasnet1_04.380.650.330.34--
mnasnet1_36.281.080.540.56--

The above results were produced using the scripts/benchmark.py script.

Note: receptive field computation is currently only valid for highway nets.

What else

Documentation

The full package documentation is available here for detailed specifications.

Example script

An example script is provided for you to benchmark torchvision models using the library:

python scripts/benchmark.py

Credits

This project is developed and maintained by the repo owner, but the implementation was inspired or helped by the following contributions:

Citation

If you wish to cite this project, feel free to use this BibTeX reference:

@misc{torchscan2020,
    title={Torchscan: meaningful module insights},
    author={François-Guillaume Fernandez},
    year={2020},
    month={March},
    publisher = {GitHub},
    howpublished = {\url{https://github.com/frgfm/torch-scan}}
}

Contributing

Any sort of contribution is greatly appreciated!

You can find a short guide in CONTRIBUTING to help grow this project!

License

Distributed under the Apache 2.0 License. See LICENSE for more information.

Rate & Review

Great Documentation0
Easy to Use0
Performant0
Highly Customizable0
Bleeding Edge0
Responsive Maintainers0
Poor Documentation0
Hard to Use0
Slow0
Buggy0
Abandoned0
Unwelcoming Community0
100