Statistics of Linux Kernel development Tsugikazu SHIBATA NEC - PowerPoint PPT Presentation

Statistics of Linux Kernel development Tsugikazu SHIBATA NEC Oct.22 nd 2009 Japan Linux Symposium @Akihabara Convention Hall

$ who am i ● Name : Tsugikazu SHIBATA ● Company: OSS promotion center NEC ● Job: Standing in between the community and industry, fostering industry's developer to work with community ● Maintaining: Documentation/ja_JP/HOWTO etc ● Japanese translated version of HOWTO and other documents merged in 2.6.23 ● Providing Google calendar feed of Linux kernel release history 2

Source code growth of Linux ● Now, Linux has 11.9ML with 409K files in 2.6.31. ● Growing 18%, about 1.85ML in this year(=last 4 versions) ● 4-5 releases per year ● Over 10ML of codes since 2.6.28 ● 6 years since 2.6.0 (was released in 2003/12/18) [ 2009 ] [ 2005 ] [ 2006 ] [ 2007 ] [ 2008 ] [ 2004 ] 3

Development days for release ● Average of last 4 releases: 83 days ● Average of last 10 releases: 86 days 4

Number of developers sent patch ● Patches from 1291 developers were merged in 2.6.31 ● About twice compared with 2.6.15 (early 2006) 5

Send patch and mailing list ● 141 mailing lists in VGER.KERNEL.ORG. ● Subscribers in Linux-kernel mailing list are 5491. ● Lots of subsystem mailing lists are also there. ● Total subscribers of 141 mailing lists are 31594. Developers Subsystem Mailing list ... Subsystem Maintainers Linux release 6 Linux-kernel Mailing list

Number of MAINTAINERS ● Number of Maintainers are increasing with the source code growth. ● 767 maintainers in 2.6.31 ● it is twice with 2.6.10 as same as source lines become twice 7

Linux kernel development Community 1: Linus 10-15: Core maintainers 80-90: kernel Summit maintainers 760: MAINTAINERS 1300: Developers 5400: LKML subscribers 31000: Linux subsystem mailing lists subscribers 8

Development Process ● Merge window and Release candidates ● 2 weeks of merge window just after kernel released ● Each RC takes a week or 10 days, continue 8-9 times Week: 0 2 11-12 Merge Window 2.6.N 2.6.N+1 -rc1 -rc2 -rc3 -rc.. New features Stabilize Kernel 2.6.26 2.6.27 2.6.28 2.6.29 2.6.30 2.6.31 Max -RC -rc9 -rc9 -rc9 -rc8 -rc8 -rc9 9

Linux-next and stable releases ● Linux-next: ● Merging 144 of subsystem trees to prepare next merge window ● Do automated build test and maintainers get notification about build problems on almost daily basis 2.6.N 2.6.N+1 -rc1 -rc.. Linux-next 10

Linux-next and stable releases ● Stable releases: ● Stable version of Linux kernel ● Including bug fixes and security fixes ● 4 digits' name and started from 2.6.11.1 ● Continue until the next new version released (at least) -rc1 2.6.N -rc.. 2.6.N+1 Stable release Linux-next 11

Source code growth ● 2-5% growth of lines in version (average: 584KL) ● 1.85ML grown in the total of the last 4 versions Lines of Difference % Lines of Difference % Code w/ prev. ver. Code w/ prev. ver. 2.6.11 6624076 128534 2.0% 2.6.21 8246517 143984 1.8% 2.6.12 6777860 153784 2.3% 2.6.22 8499410 252893 3.1% 2.6.13 6988800 210940 3.1% 2.6.23 8566606 67196 0.8% 2.6.14 7143233 154433 2.2% 2.6.24 8859683 293077 3.4% 2.6.15 7290070 146837 2.1% 2.6.25 9232541 372858 4.2% 2.6.16 7480062 189992 2.6% 2.6.26 9411790 179249 1.9% 2.6.17 7588014 107952 1.4% 2.6.27 9630023 218233 2.3% 2.6.18 7752846 164832 2.2% 2.6.28 10115662 485639 5.0% 2.6.19 7976221 223375 2.9% 2.6.29 10930802 815140 8.1% 2.6.20 8102533 126312 1.6% 2.6.30 11557329 626527 5.7% 2.6.31 11966482 409153 3.5% 12

Patch example ● Patch includes changes for lines ● Remove and add with line by line basis --- a/Documentation/ja_JP/HOWTO +++ b/Documentation/ja_JP/HOWTO @@ -412,13 +420,32 @@ Linux-kernel メーリングリストで収集された多数のパッチと同 bugzilla.kernel.org は Linux カーネル開発者がカーネルのバグを追跡する 場所です。ユーザは見つけたバグの全てをこのツールで報告すべきです。 どう kernel bugzilla を使うかの詳細は、以下を参照してください - - http://test.kernel.org/bugzilla/faq.html - + http://bugzilla.kernel.org/page.cgi?id=faq.html 13

Add and delete Lines by the patches ● Add and delete lines by the patches are 1.6ML-2.2ML ● It is about 10-20% of the source lines. Lines of Patch % Lines of Patch % Code add+del Code add+del 2.6.13 6988800 687287 10.1 2.6.23 8566606 889418 10.5 2.6.14 7143233 565473 8.1 2.6.24 8859683 1533400 17.9 2.6.15 7290070 918325 12.9 2.6.25 9232541 1354956 15.3 2.6.16 7480062 657461 9.0 2.6.26 9411790 1143004 12.4 2.6.17 7588014 797260 10.7 2.6.27 9630023 2311242 24.6 2.6.18 7752846 656463 8.7 2.6.28 10115662 1634825 17.0 2.6.19 7976221 909700 11.7 2.6.29 10930802 2233640 22.1 2.6.20 8102533 446251 5.6 2.6.30 11557329 1839720 16.8 2.6.31 11966482 1622731 14.0 2.6.21 8246517 542521 6.7 2.6.22 8499410 885487 10.7 14

Summary ● Linux 2.6 development is now 6 years since 2004. ● People sent patch are about 1300 and it is twice compared with 3 years ago. ● The number of maintainers becomes twice and it's now 760. ● 4-5 kernel version were released in a year, with 18% growth rate(1.85ML added). ● 5500 people are subscribing Linux-kernel mailing list. ● 140 of related mailing lists and total subscribers are about 31000. 15

Summary ● Linux is now over 10ML of source code since early this year. ● 2.6.31, the latest kernel has 12ML of source code in 409K of files. ● Release cycle of Linux is about 83-86 days with 8-9 times of release candidates. ● 3-5% of source code, 580KL are growing in each version. ● 10-20% of source code,1.6ML – 2.2ML are modified in each version by the patches. 16

Does Linux bloated and huge? In the discussion about performance drops on database benchmark, Linus said ”kernel is getting bloated and huge, yes it's problem” . Also “it's unavoidable due to the new features get added with each releases” . From http://lwn.net/Articles/354835/ $ pwd /home/ts/linux-2.6 $ ls COPYING Makefile crypto init net tools CREDITS README drivers ipc samples usr Documentation REPORTING-BUGS firmware kernel scripts virt Kbuild arch fs lib security MAINTAINERS block include mm sound 17

Source line share of 2 nd dirs. ● Major 7 directories take 95%. ● This mix has not been changed since 3 years ago. 100% 3% 3% 3% 3% 3% 3% 3% 3% 3% 3% 2% 2% 4% 3% 4% 3% 90% 6% usr 9% 9% 10% 10% 9% 9% 9% 4% 4% 4% 5% 5% samples 5% 5% 5% 5% 6% 5% 5% 5% 5% 5% 5% 80% 6% virt 6% 6% 6% 8% 6% 6% 8% 8% 6% 6% 8% init 8% 70% 8% 8% ipc 8% 9% 9% 8% 8% . 23% 23% 23% 60% block 24% 23% 20% 20% 20% 20% 20% 20% 20% lib 50% firmware security crypto 40% scripts mm 30% kernel 50% 50% 49% 47% 46% 46% 46% 46% 45% 45% 45% 45% Documentation 20% include sound 10% net fs 0% 2.6.20 2.6.21 2.6.22 2.6.23 2.6.24 2.6.25 2.6.26 2.6.27 2.6.28 2..6.29 2.6.30 2.6.31 arch 18 drivers

Growth of configuration file ● # of Kconfig files now 610 (2.6.31) ● About 40 files were added in each of the last 4 versions. 19

Growth of config items ● Now, # of config items are 9200 in 2.6.31 ● Config items are growing 300-400 in each version ● About 3-4% growth in each of the last 4 versions 20

In between the Performance and Features ● Seems no specific bloat in the level of subsystem directories. Mix of source lines of directories were not changed. ● Choosing good configuration option may be a solution. 21

Linux upstream and distros ● Distros take 6-7 months from upstream Kernel release to provide commercial products for it's integration and testing. ● New versions releasing every 2-3 years. 2004 2005 2006 2007 2009 2008 RHEL4 2.6.9 RHEL5 2.6.18 Feb. Oct. Mar. Sep. Fedora3 Fedora6 Dec. Oct. 2.6.27 SLES11 SLES10 2.6.16 Mar. Oct. Jul. Mar. OpenSUSE11.1 SUSELinux10.0 Dec. May. 22

My some suggestions to industry ● Creating your own Linux will never be realized. ● Your own feature will become obsolete. Similar feature will be implemented by more excellent approach by the community. ● Because of 1300 people VS your team ● Adding your own feature to Linux for your customer? ● You will see the problem for bug/security fix in the future. ● Huge number of lines are changed (10-20% and 1.6- 2.2ML by the patches) in a year. ● Having your own patch for your customer? ● You should try to merge to upstream , otherwise you will have to make your own patch forever. 23

My other suggestions ● If you need to decrease the QA engineering costs, ● Assign someone to work for QA activity with the community. ● Better quality in earlier phase leads smaller cost in later phase ● If you need more knowledge of Linux for customer support ● Assign someone to work for the community. ● He can create human network and get much information for customer support. ● If you need to educate your engineer, ● Assign someone to work for the community ● Your engineer will get good suggestions and much more technical skills from many of great people in the community. 24