Scientific literature

1. Understanding the field: second language acquisition and applied linguistics

📖 Ellis, R. (2021). A short history of SLA: Where have we come from and where are we going? Language Teaching. https://doi.org/10.1017/S0261444820000038

Key terms:

• Applied Linguistics (AL)
  • in theory: linguistics in many social applications, incl. forensic linguistics, etc.
  • often: applied only to language learning and teaching
• Second Language Acquisition (SLA)
  ≃ Foreign/Second Language Learning
• Language Teaching
  ≃ Instructed Second Language Acquisition
• Computer-Assisted Language Learning
  ≃ Language Learning & Technology
  ≃ Technology-enhanced Language Learning (and other terms)
• Language Testing / Language Assessment

Differentiating academic documents

• Book
  • Edited book (e.g., Handbooks)
  • Chapter/section in an edited book
• Journal article
• Paper in conference proceedings
  • Unpublished presentation at a conference
• Theses and dissertations
• Working papers and reports

Main journals in the field

See my List of journals in linguistics and language learning

→ Browsing journals to get an idea of the field

See also the Open Accessible Summaries in Language Studies (OASIS) Database.

2. Finding and accessing scientific literature

Searching for scientific literature

Search engines

• Elicit: AI-powered search engine for research questions, which summarises the main findings of each paper.
• Dimensions (>130M publications): useful search filtering tools (year, categories, authors…), useful citation statistics to rank publications by impact. Volume (130M) makes it a relatively good free alternative to Scopus and Web of Science.
• Google Scholar (>400M publications): most complete (the best if you want exhaustiveness), but a lot of noise (low-quality papers) and suboptimal ordering.
• SemanticScholar (>200M): better ordering and interesting insights on citation flow + helps you obtain the PDF, but incomplete citation data.
• ResearchRabbit: interesting AI-based search to start a literature review by expanding from one source; also useful to keep track of new publications (scientific watch).

Scientific databases

In comparison with search engines, databases include less publications. They have more strict inclusion criteria for journals and publications, which tend to leave out less important publishers, languages and documents.

Main limitation: the most important databases (Scopus and Web of Science) require a paid subscription by your university. They also favour journal articles, and hence empirical studies, over books and book chapters, which are an important source of literature reviews.

Especially useful if needing to establish a robust, finite and replicable set of publications (e.g., for a research synthesis). They might also help you identify the most important/reliable sources at the beginning of your research journey.

• Scopus ⭐ (~80M publications)
• Web of Science ⭐ (~85M)
• EBSCOHost (~132M)
• ProQuest (~280M)

Regional databases, focused on specific languages (Spanish, French…) and disciplinary databases, focused on specific subjects, have the major disadvantage of being smaller by multiple orders of magnitude and thus only including a very small portion of the available research.

• ERIC (~1.7M publications): focused on education, but might not include all journals on language learning (and does include some predatory journals).
• Dialnet (~8M): focus on Spain.
• Redalyc (~0.8M): focus on Mexico & Lat. Am.
• Scielo (~0.5M): focus on Brazil & Chile.

Organizing papers: reference management

Reference management software

• Zotero (strongly recommended)
  • advantages:
    • auto import citation data from websites (with Zotero Connector)
    • auto identification of PDF files (finds good citation metadata based on the PDF, but still needs some manual check afterwards)
    • integration with Word
    • free & open source
    • powerful, endless possibilities of tagging, notes, exports, collaboration (with Zotero Groups)
  • also install the Zotero Connector browser extension (for Firefox, Chrome, Safari or Edge)
  • Quick start guide
  • Recommended extensions:
    • Better BibTeX for Zotero if you use LaTeX (not necessary for most people).
    • Attanger or Zotfile to organize PDF files on your computer.
    • scite for citations count; otherwise Zotero Citation Counts Manager.
  • Recommended configuration:
    • Add column “Date Added” (in More columns) to order items by last added: this way you see on top the last publications you have found.
    • Create a free account on Zotero and use it to sync your library.
      • If you have many PDFs, use Attanger or Zotfile to put the PDFs in a specific folder, avoiding syncing those via Zotero (saving space).
    • If you collaborate on a project, create a Group and a shared library.
• Mendeley (also quite good, but less powerful than Zotero, and proprietary, owned by Elsevier).

One-time bibliography generation:

• ZoteroBib
• CiteThisForMe

Accessing papers

How to access full-text papers? (e.g., in PDF)

Many quality papers from international journals are sadly behind paywalls: if your university does not have a subscription (which is very rare in Ecuador), you have to pay for the PDF. Do not pay for one paper! It’s too expensive for a single publication, especially if you’re not sure how good or relevant it’s going to be. And authors don’t get a single cent from scientific publications.

There are solutions:

• Open access journals/papers: some papers are free to download (🔓).
• Repositories: many authors now publish a full version (often a preprint) on a personal website or an institutional repository (a website from their university which contains all publications by the university’s faculty), or on “academic social networks” such as ResearchGate.
  • SemanticScholar is pretty good at finding freely available versions of publications: search for the full title and see if there is a PDF link.
• University library and proxy access for some subscribed content:
  • Access the publisher’s website through your university proxy, if your university has a subscription (sadly very rare in Ecuador and the Global South, because too expensive for most universities).
• As a last resort, e-mail the first author asking for the PDF: most people will gladly send it to you.

Not fully legal solutions:

• Sci-Hub for papers (not legal, but considered legitimate by the large majority of the scientific community).
  • Search for the DOI, URL or full title of the paper.
• Library Genesis for books (more problematic in terms of copyright infringement than Sci-Hub).
  • You won’t find all books, so search by keywords to see what is available.

Prioritising the literature

Identifying key sources (when starting researching a topic)

Criteria:

• Citation count/year
  • When comparing publications’ citation count, normalize by year since publication: a paper from last year with 10 citations is actually more influential (10 citations/year) than one from 20 years ago with 100 citations (5 citations/year).
• Recentness
  • Most recent papers will refer to other important recent publications.
• Tools
  • SemanticScholar: Highly influential papers
  • ResearchRabbit can help to identify most cited/relevant papers
  • Elicit
• Snowballing: start from a few key papers and look at their references.

Categorizing sources

J. Hayton, Day 12: How to filter the academic literature (2018)

• A: best + most relevant → read and re-read
• B: high quality + relevant, but not essential
• C: maybe interesting (ok quality, not clearly relevant) → keep for later
• D: not relevant at all / low quality → throw away/do not spend time

3. Summarizing and reviewing the literature

Reading and taking notes for writing

📖 Lundin, E. (2021). How to write a killer research paper (even if you hate writing) [blog post]. College Info Geek. https://collegeinfogeek.com/how-to-write-a-research-paper/

• Skimming and identifying relevant information
• Synthesis matrix (for contrasting theoretical perspectives)
• Summary table of studies (for summarizing empirical studies)
• Outlining for note-taking
  • Outlining software:
    • Obsidian (recommended)
      • Use your existing folder architecture (e.g., /Thesis/… or /Research/…) as your vault.
    • Workflowy
    • Notion
    • Microsoft OneNote (not recommended)
• Preparing to write
  • Hierarchical outline
  • Flat outline

Summarizing previous research

Summarizing theories

Synthesis matrix

A synthesis matrix is used to contrast theoretical perspectives on different issues (2-3, max. 4, different authors, traditions or papers). Think for example “Chomsky vs. Saussure’s linguistics” or “Krashen vs. Ellis’ vision of input and output”. It works well when the selected authors or traditions (i.e., groups of authors) have very clear divergences (and eventual convergences) on specific aspects. Every column is one author or tradition or theoretical perspective. Every row is an aspect of the topic on which the authors either agree or disagree.

Summarizing empirical studies

• Understanding empirical results
• Statistical significance and p-value
• Effect size
• Meta-analysis

Summary Table of Studies

A Table of studies works well for summarizing the main characteristics and findings of empirical studies, whether qualitative or—most often—quantitative. Many studies can be included (from 5 to 25, 40, 100…). Every line is one study, represented by one publication (reference).

Example:

Other random examples:

Advanced example (in medical research), with extended study information : NIH summary of clinical trials on Ivermectin as a treatment for COVID-19

In such presentation of studies, information is organized in a title and 3 categories:

• Study key information (Title)
  • Study design
  • Location
  • Sample size
• Methods
  • Key inclusion criteria
  • Key exclusion criteria
  • Intervention(s)
  • Primary endpoints (+ secondary, + safety endpoints)
• Results
  • Number of participants
  • Participant characteristics
  • Primary outcomes
  • Secondary outcomes
  • Safety outcomes
• Interpretation
  • Key limitations
  • Interpretation

Identifying gaps

Citing and quoting

Literature review

How to write a literature review

📘 Frederiksen, L., & Phelps, S. F. (2020). Literature reviews for education and nursing graduate students. Rebus Community. Read online PDF

Examples of literature reviews in SLA and education

• Restrepo Ramos, F. D. (2015). Incidental vocabulary learning in second language acquisition: A literature review. Profile, 17(1), 157–166. https://doi.org/10.15446/profile.v17n1.43957 PDF
• Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(1), 7–27. https://doi.org/10.1558/cj.v21i1.7-27
• Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-based learning: A review of the literature. Improving Schools, 19(3), 267–277. https://doi.org/10.1177/1365480216659733 PDF
• Barrot, J. S. (2021). Social media as a language learning environment: A systematic review of the literature (2008-2019). Computer Assisted Language Learning, in press. https://doi.org/10.1080/09588221.2021.1883673
• Ellis, R. (2005). Instructed second language acquisition: A literature review [Report to the Ministry of Education]. Research Division, Ministry of Education. PDF
• Engh, D. (2013). Why use music in English language learning? A survey of the literature. English Language Teaching, 6(2). https://doi.org/10.5539/elt.v6n2p113
• Mitchell, A., & Savill-Smith, C. (2004). The use of computer and video games for learning: A review of the literature. Learning and Skills Development Agency. PDF

Research synthesis vs. literature review

📄 Li, S., & Wang, H. (2018). Traditional literature review and research synthesis. In A. Phakiti, P. De Costa, L. Plonsky, & S. Starfield (Eds.), The Palgrave Handbook of Applied Linguistics Research Methodology (pp. 123–144). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-59900-1_6 PDF

Examples of research syntheses

• Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college‐level L2 writing. Applied Linguistics, 24(4), 492–518. https://doi.org/10.1093/applin/24.4.492
• Chong, S. W., & Reinders, H. (2020). Technology-mediated task-based language teaching: A qualitative research synthesis. Language Learning & Technology, 24(3), 70–86. https://doi.org/10125/44739
• Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877. https://doi.org/10.1080/09588221.2018.1535508 PDF
• Sauro, S. (2011). SCMC for SLA: A research synthesis. CALICO Journal, 28(2), 369–391. https://doi.org/10.11139/cj.28.2.369-391
• Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning, 29(2), 365–397. https://doi.org/10.1080/09588221.2014.960942
• Vandewaetere, M., Desmet, P., & Clarebout, G. (2011). The contribution of learner characteristics in the development of computer-based adaptive learning environments. Computers in Human Behavior, 27(1), 118–130. https://doi.org/10.1016/j.chb.2010.07.038

Examples of meta-analyses (research synthesis + statistical aggregation)

• Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3), 417–528. https://doi.org/10.1111/0023-8333.00136
• Ziegler, N. (2013). Synchronous computer-mediated communication and interaction: A research synthesis and meta-analysis [PhD dissertation, Georgetown University]. PDF
• Saito, K., & Plonsky, L. (2019). Effects of second language pronunciation teaching revisited: A proposed measurement framework and meta-analysis. Language Learning, 69(3). https://doi.org/10.1111/lang.12345 PDF
• Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second language anxiety and achievement: A meta-analysis. Studies in Second Language Acquisition, 41(2), 363–387. https://doi.org/10.1017/S0272263118000311
• Bibauw, S., Van den Noortgate, W., François, T., & Desmet, P. (2022). Dialogue systems for language learning: A meta-analysis. Language Learning & Technology, 26(1). PDF