The state of packaging Tarek Ziad tarek@critsend.com @tarek_ziade - PowerPoint PPT Presentation

The state of packaging Tarek Ziadé tarek@critsend.com @tarek_ziade Friday, February 19, 2010

Google Moderator http://tiny.cc/packaging13 Friday, February 19, 2010

Packaging sucks Friday, February 19, 2010

2007 We need to package and release our Python apps ! on all systems ! I’ll work on it ! Packaging ? mm.. Distutils or Setuptools should work Friday, February 19, 2010

So 2007 how do we install our product ? easy_install foo Sweet ! How do we remove it ? Manually Are you kidding ? But boss, you want a portable installer ! Friday, February 19, 2010

2008 And create local installation I’ll use virtualenv ! with everything For every app sysadmins will hate me Friday, February 19, 2010

2008-9 But I want to This will be my improve packaging ! contribution to Python Let’s try to fix things from the root Friday, February 19, 2010

Distutils Setuptools ? Developer PoV Friday, February 19, 2010

python setup.py pip install foo easy_install foo install site-packages User PoV Friday, February 19, 2010

Distutils 4suite.distutils numpy.distutils Setuptools PyPM Distribute Enstaller Pip Global PoV Friday, February 19, 2010

Standardization Friday, February 19, 2010

What is presented 1. Metadata standardization 2. Version schemas 3. Installed distributions index Friday, February 19, 2010

What is not presented • build/compiling flaws in Distutils • target-specific binary distributions problems • trillions of things we are working on... Friday, February 19, 2010

Distutils is not cmake Friday, February 19, 2010

Part 1 Metadata standardization Friday, February 19, 2010

Actual PEP 314 Metadata for Python Software Package 1.1 Friday, February 19, 2010

PKG-INFO example M e t a d a t a - V e r s i o n : 1 . 0 N a m e : p i p V e r s i o n : 0 . 6 S u m m a r y : p i p i n s t a l l s p a c k a g e s . P y t h o n p a c k a g e s . A n e a s y _ i n s t a l l r e p l a c e m e n t H o m e - p a g e : h t t p : / / p i p . o p e n p l a n s . o r g A u t h o r : T h e O p e n P l a n n i n g P r o j e c t A u t h o r - e m a i l : p y t h o n - v i r t u a l e n v @ g r o u p s . g o o g l e . c o m L i c e n s e : M I T K e y w o r d s : e a s y _ i n s t a l l d i s t u t i l s s e t u p t o o l s e g g v i r t u a l e n v P l a t f o r m : U N K N O W N C l a s s i fi e r : D e v e l o p m e n t S t a t u s : : 4 - B e t a C l a s s i fi e r : I n t e n d e d A u d i e n c e : : D e v e l o p e r s C l a s s i fi e r : L i c e n s e : : O S I A p p r o v e d : : M I T L i c e n s e C l a s s i fi e r : T o p i c : : S o f t w a r e D e v e l o p m e n t : : B u i l d T o o l s Friday, February 19, 2010

Distutils metadata cycle Project Distribution python setup.py sdist setup.py PKG-INFO (PEP 314) setup(options) PyPI Target system PKG-INFO PKG-INFO (PEP 314) (PEP 314) Friday, February 19, 2010

Setuptools metadata cycle Project Distribution python setup.py sdist setup.py PKG-INFO (PEP 314) setup(options) + requires.txt + .... PyPI Target system PKG-INFO PKG-INFO (PEP 314) (PEP 314) + requires.txt + ????? + .... Friday, February 19, 2010

Accepted PEP 345 Metadata for Python Software Package 1.2 Friday, February 19, 2010

Requires-Dist Requires-Dist: zope.interface (>3.5) Requires-Dist: PasteDeploy Friday, February 19, 2010

Version specifiers Requires-Dist: zope.interface (>3.5, <4.0) <, >, <=, >=, == and != Friday, February 19, 2010

Version specifiers • > 2.6, < 3 • 3.5 means 3.5.x • 2.5.0 means >=2.5.0,<2.5.1 • >=3.1,!=3.1.3,<3.2 Friday, February 19, 2010

Requires-Python Requires-Python: >2.4 Requires-Python: >=2.5, <2.7 Friday, February 19, 2010

Requires-External Requires-External: libxslt Requires-External: libpng (>=1.5) Friday, February 19, 2010

Provides-Dist Zodb Provides-Dist: transaction transaction Provides-Dist: ZODB Friday, February 19, 2010

Obsoletes-Dist Obsoletes-Dist: Setuptools Friday, February 19, 2010

Project-URL Project-URL: Bug Tracker, http://bitbucket.org/tarek/distribute/issues Project-URL: Repository, http://bitbucket.org/tarek/distribute/src Friday, February 19, 2010

Description Description: This project provides powerful math functions |For example, you can use `sum()` to sum numbers: | |Example:: | | >>> sum(1, 2) | 3 | Friday, February 19, 2010

PEP 345 - Summary • Requires-Dist * • Obsoletes-Dist * • Provides-Dist * • Requires-External • Requires-Python * • Project-URL • Description * with version specifier Friday, February 19, 2010

PEP 345 for developers Friday, February 19, 2010

PEP 345 for developers Friday, February 19, 2010

Environment markers Friday, February 19, 2010

Dynamic metadata fields Friday, February 19, 2010

Dynamic metadata PyPI MyProject.tar.gz local Pip python setup.py egg_info get dependencies Metadata python setup.py install Friday, February 19, 2010

Environment Markers Requires-Dist: value (version); marker • sys.platform == 'win32' and platform.machine == 'i386' • 'linux' in sys.platform • python_version == '2.4' Friday, February 19, 2010

Environment Markers • python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) • python_full_version = sys.version.split()[0] • os.name = os.name • sys.platform = sys.platform • platform.version = platform.version() • platform.machine = platform.machine() • a free string, like '2.4', or 'win32' Friday, February 19, 2010

Environment Markers • Requires-Python • Requires-External • Requires-Dist • Provides-Dist • Obsoletes-Dist • Classifier Friday, February 19, 2010

Pseudo-static metadata fields Friday, February 19, 2010

Pseudo-static metadata PyPI MyProject.tar.gz + metadata local Pip python setup.py install get dependencies Friday, February 19, 2010

I love PEP 345 ! Friday, February 19, 2010

Part 2 Version schemas Friday, February 19, 2010

Brought to you by PyPI 20090105 RC unreleased.unofficialdev 1.* Friday, February 19, 2010

Version 1 > Version 2 ? Distutils’ Setuptools’ LooseVersion parse_version() StrictVersion Result vary.... Friday, February 19, 2010

Distutils Setuptools >>> import pkg_resources >>> import distutils.version >>> V = >>> V = distutils.version.LooseVersion pkg_resources.parse_version >>> V('1.2rc1') < V('1.2') >>> V('1.2rc1') < V('1.2') False True >>> V('1.2-a1') < V('1.2') >>> V('1.2-a1') < V('1.2') False True >>> V('1.2a1') < V('1.2') >>> V('1.2a1') < V('1.2') False True Friday, February 19, 2010

Version scheme requirements • final version • pre-final version • post-release versions • development versions at all stages Friday, February 19, 2010

Version scheme requirements • final • MAJOR.MINOR.MICRO • pre-final • a1, a2, b1...c1, rc2, ... • post-release • +r145, -r1256, ... • development • dev12, ~dev12, -dev12, ... Friday, February 19, 2010

Accepted PEP 386 Interoperable version comparison scheme Friday, February 19, 2010

major, minor, micro extra numbers N.N[.N]+[{a|b|c|rc}N[.N]+][.postN][.devN] post- tag dev tag pre- tag N = integer Friday, February 19, 2010

< 1 . 1 1 . 0 b 2 < 1 . 0 c 1 < 1 . 0 1 . 0 a 1 < 1 . 0 a 2 < Friday, February 19, 2010

development versions 1.0a1 < 1.0a2.dev345 < 1.0b1 < 1.0.dev432 < 1.0 Friday, February 19, 2010

suggest_normalized_version(your scheme) PEP 386 compatible scheme Friday, February 19, 2010

Interoperable version tool >>> from verlib import NormalizedVersion >>> NormalizedVersion('1.2-a1') < NormalizedVersion('1.2') Traceback (most recent call last): ... raise IrrationalVersionError(s) verlib.IrrationalVersionError: 1.2-a1 >>> from verlib import suggest_normalized_version >>> suggest_normalized_version('1.2-a1') '1.2a1' >>> NormalizedVersion('1.2a1') < NormalizedVersion('1.2') True Friday, February 19, 2010

PEP 386 @ PyPI distributions 8% 3% 90% Direct match suggest_normalized_version No match Friday, February 19, 2010

We love PEP 386 Friday, February 19, 2010

Part 3 Installed distributions index Friday, February 19, 2010

Aborted draft PEP 262 A database of installed packages Friday, February 19, 2010