Scientific papers Stefano Chessa Pisa & their performances 13 - PDF document

3/13/2019 Scientific papers Stefano Chessa Pisa … & their performances 13 th March 2019 1 • Studies: PhD in Computer Science, 1999 • Past positions: researcher at the University of Pisa 2000- 2014 • Current position: associate professor at the University of Pisa About me • Since November 2015: Vice-chair of the BSc and MSc curricula in “Computer Science” of the University of Pisa • Member of the Council of the Doctorate in Computer Science since October 2013 • Supervisor of 7 PhD thesis (2 underway) • Delegate for the assessment of the quality of research for my department (since 2012) 2 1

3/13/2019 • You are a PhD student, you are learning how to conduct a research • Your supervisor is you first reference: Preliminary • He/she is experienced, and he/she know the rules of the game notes about • Learn from him as much as you can this talk This seminar is not intended to replace him/her! 3 In the last years “aggressive” use of bibliometrics to evaluate the research Why this talk … and consequent use of “aggressive” strategies by the researchers to improve their bibliometric indexes… 4 2

3/13/2019 Number of papers… # of publications in Italy and Spain From Scopus, queries: 120000 AFFILCONTRY(Italy) 100000 AFFILCOUNTRY(Spain) 80000 60000 Moore’s Law for papers: 40000 the number of papers that are “inexpensively” 20000 produced doubles every 0 10 years… 1995 2000 2005 2010 2015 2020 Italy Spain 5 Happened in 2012… # of publications in Italy and Spain • In Italy the rules for 120000 recruitment changed 110000 drastically 20K! • Pre-selection based on 100000 citations, h-index, 10K 90000 #papers 80000 • That’s explain the growth 70000 in Italy after 2012 60000 • A “ speculative bubble ”… 2008 2010 2012 2014 2016 2018 Italy Spain 6 3

3/13/2019 Part II: performance indicators & evaluation of research 7 Properties of a paper 8 4

3/13/2019 Performance v. s. Maturity venue year citations venue year citations J. of Algo. 2002 27 ComCom 2007 893 IEEE TIT 2012 9 INFOCOM 2005 197 SP&E 2010 22 ComCom 2001 121 IEEE TC 2001 13 SRDS 2001 118 9 Sometimes they are called “quality” indicators (in Italy for example) … but they are not. They Performance measure the performance of a paper or of a journal in indicators terms of “diffusion” in the research community Many different indexes: • Impact factors • H-index • Number of citations • Number of papers 10 5

3/13/2019 IF (web of science) SJR (scopus) Impact factors: SNIP (scopus) performance of CITESCORE journals (scopus) MCQ (MathSciNet, for mathematics) … 11 • “the Impact Factor of a journal is IF (web calculated by dividing the number of current year citations to the source items published in that journal during of the previous two years” science): • Example: X papers published in 2015 and 2016; Y citations received by these papers in 2017; IF 2017 =Y/X Impact factors Citescore • Equivalent to IF but computed over the scopus database (Scopus) 12 6

3/13/2019 “ SCImago Journal Rank measures weighted citations received by the serial. Citation weighting SJR depends on subject field and prestige (SJR) of the (scopus): citing serial.” Ispirato al PageRank di google Impact factors “ Source Normalized Impact per Paper measures SNIP actual citations received relative to citations (scopus): expected for the serial’s subject field.” 13 high diffusion, many readers High Impact high chance of being read & cited Publishing in more selective, harder to publish high impact • In many areas the impact of the journals is taken journals rather seriously • … and recently also for computer science & engineering it is becoming important 14 7

3/13/2019 • It’s your preliminary choice Publishing in • … but look first at the meaningfulness of the high impact journal for your paper journals (II) • and review process may be engaging… 15 The impact of my favorite journals is low! Ranking of journals of area «theoretical computer science» based on CiteScore 16 8

3/13/2019 The impact of my favorite journals is low! Ranking of journals of area «hardware & architecture» based on CiteScore 17 The impact of my favorite journals is low! Ranking of journals of area «software» based on CiteScore 18 9

3/13/2019 • However, high impact large number of citations • … why so? Publishing in high impact • The citations received by a paper are an individual value journals (III) • The impact of a journal is a collective value • All high-impact journals have highly-cited and normally/lowly-cited papers 19 • Usually, the number of citations received and the H-index are considered in combination with the journal’s impact Citations and H-Index • They indicate the “individual” performance of a researcher or of a paper 20 10

3/13/2019 • H-index of a researcher is X if he has exactly X papers each of which received at least X citations Citations and • H-index grows slowly and it is not linear! H-Index • 1 < 5 but 11 << 15 <<< 19 … • There are criticisms to H-Index, but it is still widely used 21 are usually a factor of stress and depression : • They do not (necessarily) depend on the quality of your work Citations and H-Index • They do not (necessarily) depend on your preliminary choice (as impact factors) • They depend on the future behavior of other researchers, out of your control 22 11

3/13/2019 How to get cited? there’s no guarantee, depends on many factors … and may take time… 23 Why do you cite a paper? • To avoid proving something (you cite a • To refer a work strongly related to yours paper that already proves what you • To motivate the importance of a need) research field • To defend your settings in your • To explain the impact of your research simulations on the society • To defend your approach/methodology • To avoid citing many weakly related • To defend a statement in your paper papers (you may cite a survey) • … 24 12

3/13/2019 About usefulness • Writing papers useful for a research • Sometimes we write papers just to: community is not easy • to test our ideas, • Many time you know later whether they • receive opinions from reviewers, are really useful • document our work • I don’t know of anybody who wrote only • … and sometimes even to witness or to useful papers strengthen a cooperation • In fact, most papers have a limited “usefulness” … 25 1. usefulness 2. venue Main factors 3. reputation of the authors for citations 4. size of research community 5. timeliness of the work 26 13

3/13/2019 • Not only a matter of impact • The content of the paper should match well the audience of the journal/conference 2. Venue of • Write the paper for that journal the • Use terminology, methodology, approach typical of that community publication • i.e. if they expect formal proofs give them formal proofs • If they expect simulations give them simulations • … etc… 27 Two papers with a similar idea about routing protocols in ad hoc networks, (almost) same year 2. Venue of • GPSR: Greedy Perimeter Stateless Routing the for wireless networks publication : MOBICOM 2000 – 4940 citations example • Routing with guaranteed delivery in ad hoc wireless networks Dial-M '99 – 559 citations • Later appeared also in Wireless Networks ’01 – 781 citations 28 14

3/13/2019 • You are in the best position to assess your work: • if you feel it is very good makes sense to write it for a top journal/conference • … otherwise it may be a good idea to write it anyway and address a minor venue About venue • … but write it for the venue you chose • A good venue always help good papers… • ... but it doesn’t help poor performing papers 29 Two papers with a very similar idea about routing protocols in ad hoc networks, same year 2. Venue • Virtual ring routing: Network routing & inspired by DHTs 3. Reputation: ACM SIGCOMM ‘06 – 150 citations example • Reliable routing in wireless ad hoc networks: The virtual routing protocol J. of Network and Systems Management ‘06 – 12 citations 30 15

3/13/2019 How do you gain reputation? 1. Writing high-quality papers 2. Being involved in a research community • serve the community 3. Reputation • take part to the public events • … of the 3. Being proactive in innovation: • proposing new themes of research authors • proposing new workshops/special issues • … 4. Establishing a network of connections 31 4. Size of a research community First, my main research areas… IoT MSN/ crowdsensing Human activity rec./AAL/e-health Indoor Localization Wireless sensor networks Ad hoc networks System-level diagnosis PhD in C.S. 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 32 16