Approximately two million individuals die from respiratory cancer every year (1). Non-small cell lung cancer (NSCLC), the leading pathological type of respiratory malignancy, accounts for 80% or more of respiratory cancer deaths in both sexes (2). Chemotherapies including immune checkpoint inhibitors and molecular targeted therapies play an important role in the treatment of incurable locally advanced, metastatic, and recurrent NSCLC. When relapse occurs after receiving first-line chemotherapy, a patient who maintains a reasonable performance status usually receives second-line chemotherapy. The efficacy and safety of chemotherapy in cancer patients are usually evaluated through phase I, II, and III trials. Once a phase I trial determines the dosage by safety assessment, a phase II trial is designed to test both the safety and efficacy of the treatment in a larger sample size. The best treatment efficacy outcome is overall survival (OS) as it is easy to interpret, indicates an ultimate benefit of a patient, and is not affected by an observational bias (3). However, researchers often select other indexes such as the objective response rate (ORR) and disease control rate (DCR) based on the Response Evaluation Criteria in Solid Tumors (RECIST) defined as primary endpoints of NSCLC phase II trial (3,4). Besides, ORR may be a reasonable outcome to assess the potential anticancer activity of chemotherapy (5). The RECIST is a simple and useful tool that can be used to categorize patients with solid tumor who underwent chemotherapy into four groups: complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD). Currently, the revised RECIST guidelines are widely used in the clinical and research settings of a variety of solid tumors (5).
Although ORR and DCR are commonly used as primary endpoints (5), it remains unclear whether ORR and DCR correctly reflect the OS in a NSCLC phase II trial evaluating a non-first-line chemotherapy. ORR might be easily affected because the small number of patients in each trial achieved complete or partial response in the phase II non-first-line setting. In this study, we aimed to evaluate the association of trial-level ORR and DCR with OS in randomized phase II NSCLC trials that evaluate second- or later-line chemotherapy. We present the following article in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting checklist (available at http://dx.doi.org/10.21037/tlcr-20-1120).
The current systematic review did not utilize a meta-analysis to aggregate data; however, this study was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses as it is a reasonable method of performing a systematic review of RCTs (Table S1) (6). The protocol, UMIN000040412, has been registered on the website of the University Hospital Medical Information Network Clinical Trials Registration (7).
We systematically searched PubMed, Cochrane database, Embase, and Web of Science as of May 15, 2020 (6). The search formulas are presented elsewhere (Table S2). The reference list of all included articles was also manually checked. Two investigators independently screened the titles and abstracts and they also scrutinized the full text (HM and NH). Repeated use of the same patient in duplicated studies was prohibited. If conflicts arise between the review authors during the selection of the final study, a discussion was made to resolve the inconsistencies.
Study selection, design
We included a phase II RCT that evaluated second- and/or later-line chemotherapy for advanced, locally advanced, and recurrent NSCLC (8,9). An article had to be written as a full article, a brief report, or a conference abstract regardless of its primary end point. Non-English language reports were excluded. Randomized phase I/II trials were included. The protocol permitted a phase II/III trial when the data from phase II part could be extractable, but such a trial was not found.
Study selection, patient
Patients with a pathologically or cytologically confirmed diagnosis of locally advanced, advanced, and recurrent NSCLC who relapsed after one or more of previous chemotherapies were included. No limitation was set for age, sex, smoking history, first-line chemotherapy regimen, response to the first-line treatment, status of driver mutation, performance status, and pathological subtype of NSCLC.
Study selection, treatment
A patient should be treated with any type of chemotherapeutic regimen including immune checkpoint inhibitors, molecular targeted therapy, cytotoxic agents, single drug regimens, and multi-drug regimens as non-first-line chemotherapy. Antibiotics and immunotherapy other than immune checkpoint inhibitors were not included because such treatment had never been considered as a standard therapy (8,9). Comparison of the same drugs (low-dose versus high-dose, weekly versus tri-weekly regimens, and chemotherapy versus placebo) was allowed. Maintenance therapy after first-line chemotherapy was not considered as a second-line treatment as it was administered in a patient who did not experience a relapse. A trial with treatment crossover after the first-line therapy was excluded because such a trial randomized a patient prior to the first-line treatment.
Assessment of the risk of bias
The quality of each RCT was assessed using the six domains of Cochrane risk of bias (10).
The correlation, with hazard ratio (HR) of OS (HRos), each for odds ratio (OR) of ORR (ORorr), difference of ORR (ΔORR, %), OR of DCR (ORdcr), and difference of DCR (ΔDCR, %) was evaluated. ΔORR was calculated by subtracting the ORR estimated for the second treatment arm from that estimated for the first arm. Similarly, ΔDCR was obtained. CR, PR, and SD needed to be evaluated in line with the RECIST 2000 and the revised RECIST 2009 revised guidelines (5). When an article reported that the RECIST outcome was judged by both investigator team and an independent radiographic review panel, the data of the external panel was adopted.
Data regarding the study characteristics, OS, ORR, DCR, and risk of bias were independently extracted by the two review investigators (HM and NH). Once the review authors extracted inconsistent data, a discussion is made to reach consensus. The first and second arms of each RCT were decided according to the description in each article. When a study randomized patients into three or more arms, two arms with the largest number of patients were selected for our analysis. ORR and DCR were preferably determined by full analysis set or intention-to-treat analysis; thus, a denominator to yield ORR and DCR included patients whose data were not assessable by RECIST (5). Parmar's method was applied to obtain OS data from the Kaplan-Meier curves as necessary (11). We respected the authors' judgement of disease staging regardless of the updates of TNM staging system. For example, a patient with pleural effusion without metastasis might be diagnosed with stage IIIb or IV based on the trial policy.
Spearman's rank correlation coefficient (r) was calculated using GraphPad PRISM ver 7.02 (San Diego, CA, USA). When one or more cells in the two-by-two contingency used to calculate ORorr and ORdcr were null, a continuity correction with 0.5 was applied. A correlation coefficient was interpreted as follows: |r| <0.2, no correlation; 0.2< |r| <0.4, weak correlation; 0.4< |r| <0.6, moderate correlation; 0.6< |r| <0.8, strong correlation; and 0.8< |r|, excellent correlation. The statistical significance threshold was set at P<0.05. The Begg-Kendall test was carried out to assess for publication bias. The correlation was displayed on a scatter plot.
Exploratory subgroup analyses were performed by classifying studies by characteristics such as sample size and number of responders with a median as the cutoff of subgroups.
Out of 1,437 articles that were identified by database search and hand searches, 73 met our inclusion criteria (Table S2). Since one article represented two independent RCTs, we analyzed 74 trials that randomized NSCLC patients for second- or later- line chemotherapy (Figure 1, Table 1, Appendix References).
The study characteristics of each study are summarized in Table 1, and aggregated data are presented in Table 2. The majority of these trials were reported as full-length articles that compared two arms with clear declaration of phase II except three conference abstracts, one brief report, ten three-arm trials, two four-arm trials, and one phase Ib/II trial. These 74 trials were reported from European (N=33), North American (N=21), and Asian (N=20) countries. Thirteen (18%) and two (3%) RCTs recruited patients with adenocarcinoma/non-squamous and squamous cell carcinoma, respectively, while the other 59 (80%) RCTs randomized non-small cell lung cancer patients without specifying a pathological subtype of NSCLC (Table 2). A total of 65 (88%) RCTs did not recruit patients with specific driver mutation status. The most frequently used primary endpoint was progression-free survival (PFS) (N=36, 49%) followed by ORR (N=15, 20%). Only two studies (3%) used DCR as the primary endpoint. More than two-thirds (N=51, 69%) of trials focused on second-line chemotherapy. The widely used performance status criteria were Eastern Cooperative Oncology Group (ECOG) 0-2 (N=41, 55%) and ECOG 0-1 (N=25, 34%). The median number of patients in the two arms was 116 [interquartile range (IQR), 82–159]. Single-agent docetaxel was the most frequently used treatment arm (23 arms) followed by single-agent pemetrexed (14 arms) and erlotinib alone (14 arms) probably because this regimen was used as a standard second-line regimen over the years (Table 1) (9,12). Treatments prior to each study was inconsistent. Sixty-eight studies (92%) had at least one domain of risk of bias (Table 1, Table S3). The key outcomes of each trial are also presented in elsewhere (Table S4).
ORR and DCR
Most studies described the ORR and DCR data but five only reported ORR and one only reported DCR (Table S4). Among the 146 arms from 73 studies, the median ORR was 11% (IQR, 5–17), which resulted in a median number of responders (CR+PR) in each study of 13 (IQR, 6–22) patients. The median number of patients with disease control (CR+PR+SD) in the 136 arms from 68 RCTs was 56 (IQR, 36–77). The median DCR was 51%. No publication bias was found in either the ORR or DCR (Figure S1).
Correlation between ORR and OS
The Spearman’s correlation between trial-level ORorr and HRos was weak (r=−0.29, 95% CI: −0.49 to −0.05, P=0.014, Figure 2). ΔORR also had a weak correlation with HRos (r=−0.33, 95% CI: −0.52 to −0.10, P=0.005) (Figure S2).
The exploratory subgroup analyses between ORorr and HRos suggested a prominent inter-subgroup difference in responder-based analysis. No correlation (|r|<0.2) was observed in the “responders, -10” subgroup (r=−0.15, 95% CI: −0.47 to 0.21, P=0.403, Figure 3) while that in the “responders, 11-” subgroup was moderate (r=−0.51, 95% CI: −0.72 to −0.22, P <0.001, Figure 3).
Correlation between DCR and OS
Both ORdcr (r=−0.53, 95% CI: −0.68 to −0.32, P <0.001, Figure 2) and ΔDCR (r=−0.52, 95% CI: −0.68 to −0.31, P<0.001, Figure S2) had a moderate rank correlation with HRos. Compared with ORorr, ORdcr had the same or higher-level correlation with HRos in any subgroup analyses (Figure 3).
In the assessment of the efficacy of chemotherapy in solid malignancies, tumor shrinkage judged by ORR has traditionally been regarded as a proxy of clinical benefit and prolonged survival (13) partially because the response indicates the anticancer activity of chemotherapy (5). Thus, ORR was the preferred outcome rather than DCR in the chemotherapy trials. A total of 15 (20%) out of 74 phase II trials in our analysis adopted ORR as the primary endpoint, while only 2 (3%) selected DCR (Table 2). In this study, we assessed the correlation of ORR and DCR with HRos at trial level in randomized phase II trials that evaluated second- or later-line chemotherapy for NSCLC. ORdcr (r=−0.53, P<0.001) and ΔDCR (r=−0.52, P<0.001) moderately correlated with HRos; however, ORorr (r=−0.29, P=0.014) and ΔORR (r=−0.33, P=0.005) had weak correlations (Figure 2). If ORR and DCR capture some patient-oriented benefits such as survival and quality of life, they can be useful study endpoints. Otherwise, the application of these endpoints might be questionable because no imaging modality offers a straightforward measurement of patient-oriented benefits (3,14-16). Even though a chemotherapy regimen with both favorable ORR and poor survival can pass the phase II trial that adopts ORR as the primary endpoint, such treatment cannot meet the primary OS endpoint of phase III trials. By contrast, regimens such as atezolizumab that lead to prolonged survival without good response should be highly recognized (17,18). Data on ORR and DCR are always simultaneously reported. Given the better trial-level association between DCR and survival, DCR seems to be a superior outcome in the phase II trial evaluating non-first-line chemotherapy for NSCLC. Although the current analysis included only RCTs, DCR may also be a reasonable outcome in a single-arm phase II trial. In addition to the better surrogacy of OS, broader applicability for patients without measurable disease is another advantage of DCR (13) because a trial that adopted ORR as the primary endpoint should recruit patients with at least one measurable lesion.
ORR is more frequently selected as the study endpoint compared with DCR (Table 2); however, some published studies have indicated that DCR is a more accurate surrogate of OS. Lara et al. gathered data of nearly 1,000 NSCLC patients from three RCTs for platinum-based chemotherapy (13). Although patients with response, CR+PR, led longer survival with a HR of 0.61, maintaining at least stable disease, CR+PR+SD, was a stronger predictor of OS with a HR of 0.45. Half of the trials in our review adopted PFS as the primary endpoint (Table 2). PFS resembles DCR since response is not necessary, while ORR and response duration require at least PR. Mandrekar et al. analyzed the individual data of 284 patients with NSCLC enrolled in phase II trials (4). They concluded that PFS at 12 weeks, a similar metric to DCR, more accurately predicted subsequent survival compared with tumor response. The methodology of our study evaluating trial-level association was different from that of previous studies (4,13). However, our conclusion was consistent with the conclusions of these preceding analyses. The better surrogacy of DCR was confirmed for solid tumors other than NSCLC. Lara et al. retrospectively analyzed the data of 263 extensive small-cell lung cancer patients from phase II trials in 2016 (19). In this setting, DCR (HR 0.45) was a more prominent surrogate of OS than ORR (HR 0.74). DCR was also more tightly linked to the trial-level OS of advanced colorectal cancer in the first-line setting than ORR (DCR, r=0.975, R2 =0.889; ORR, r=0.866, R2 =0.484).
We would like to consider why ORR did not reflect survival in a phase II RCT of non-first-line chemotherapy for NSCLC. First, ORR overrates “aggressive” chemotherapy, such as a high-dose regimen and a combination treatment. Aggressive chemotherapeutic treatment damages both tumor cells and normal cells, which results in good ORR and deteriorated organ function. Once ORR is chosen as the primary endpoint, a researcher might be motivated to use excessively aggressive chemotherapies. Choosing an aggressive regimen as second-line chemotherapy is especially risky because toxicity caused by first-line treatment already affected the patient's organ function. The optimal endpoint might be inconsistent between the first-line and the later-line treatments. Our previous analysis indicated that the ORR of the first-line NSCLC chemotherapy better predicts OS than DCR (3). A chemo-naive patient can endure an aggressive chemotherapy, and a chemo-naive tumor lesion is usually more responsive to an anti-tumor medication. Selecting an aggressive regimen with capable anticancer activity that yields high ORR is a feasible choice for first-line RCT but not a reasonable strategy for a later-line trial. The second possibility is that ORR might be unreliable due the small number of responders in the second- or latter-line phase II setting (19). In the explanatory subgroup analysis, no correlation was observed between ORR and OS in the “responder-10” subgroup, while moderate correlation was found in the “responder 11-” subgroup (Figure 3). DCR may be a reliable outcome because of the higher event frequency.
ORR is a popular endpoint in phase II non-first-line trials for solid tumors other than NSCLC. Nonetheless, we suspect that DCR more accurately reflects OS in these trials because patients are often exhausted after receiving first-line treatment and there are usually few responders in the non-first-line phase II trial regardless of malignancy type. The trial-level correlation between ORR and OS in phase II second-line trials should be evaluated for other malignancies such as colorectal cancer because the ORR of the second-line chemotherapy for these types of cancers is as low as that for NSCLC (20,21).
Even though DCR better reflected OS than ORR, the surrogacy of DCR was still unsatisfactory. Late PFS, such as 6-month PFS, might be a more accurate surrogate of OS than DCR in studies with a longer follow-up duration. OS data directly demonstrate the patient’s benefit, although it requires further longer follow-up to aggregate the number of events. In principle, longer follow-up is not recommended in a phase II trial as a phase II trial is just a screening process to select a regimen that will be assessed in a phase III trial. There is always a tradeoff between promptness and outcome value when selecting a phase II trial endpoint.
This study has some limitations. First, although OS is the gold standard endpoint of an advanced NSCLC trial, OS may be easily affected by later-line treatment. Nonetheless, this issue did not make our analysis unreliable because successive chemotherapy after the non-first-line treatment cannot largely provide survival benefit. Second, the current consensus recommends that NSCLC patients should be treated based on pathological subtype and driver mutation status (8,9); however, most of the included studies did not mention these data. Third, we analyzed studies adopted cytotoxic agent, MMT, ICI, and combination of them collectively. Fourth, PFS-related analysis could not be performed due to inconsistent PFS data format from the trials including 2-month PFS, median PFS, hazard ratio of PFS, P value from log-rank test. Fifth, most of studies recruited patients with inconsistent prior treatments.
In conclusion, we systematically searched for randomized phase II trials that evaluated second- or later-line chemotherapy for NSCLC. ORR is a more frequently used as a primary endpoint than DCR (Table 2). According to 68 trials, DCR had moderate trial-level correlations (ORdcr r=−0.53, ΔDCR r=−0.52) with HRos while ORorr and ΔORR had weak correlations (ORorr r=−0.29, ΔORR r=−0.33). The subgroup analysis suggested that the rarity of responders in the phase II non-first-line setting may lead to the poor association between ORR and HRos. Since data on ORR and DCR are available simultaneously, we recommend using DCR instead of ORR as the primary endpoint in a randomized phase II trial evaluating second- or later-line chemotherapy for NSCLC.
Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at http://dx.doi.org/10.21037/tlcr-20-1120
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/tlcr-20-1120). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
- Fitzmaurice C, Abate D, Abbasi N, et al. Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life-Years for 29 Cancer Groups, 1990 to 2017: A Systematic Analysis for the Global Burden of Disease Study. JAMA Oncol 2019;5:1749-68. [Crossref] [PubMed]
- Molina JR, Yang P, Cassivi SD, et al. Non-small cell lung cancer: epidemiology, risk factors, treatment, and survivorship. Mayo Clin Proc 2008;83:584-94. [Crossref] [PubMed]
- Nakashima K, Horita N, Nagai K, et al. Progression-Free Survival, Response Rate, and Disease Control Rate as Predictors of Overall Survival in Phase III Randomized Controlled Trials Evaluating the First-Line Chemotherapy for Advanced, Locally Advanced, and Recurrent Non-Small Cell Lung Carcinoma. J Thorac Oncol 2016;11:1574-85. [Crossref] [PubMed]
- Mandrekar SJ, Qi Y, Hillman SL, et al. Endpoints in phase II trials for advanced non-small cell lung cancer. J Thorac Oncol 2010;5:3-9. [Crossref] [PubMed]
- Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228-47. [Crossref] [PubMed]
- Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. [Crossref] [PubMed]
- University hospital Medical Information Network Center. University hospital Medical Information Network Center Clinical Trial Registry. Accessed on June 15, 2020. Available online: https://www.umin.ac.jp/ctr/index-j.htm.
- Reck M, Rabe KF. Precision Diagnosis and Treatment for Advanced Non-Small-Cell Lung Cancer. N Engl J Med 2017;377:849-61. [Crossref] [PubMed]
- Zappa C, Mousa SA. Non-small cell lung cancer: current treatment and future advances. Transl Lung Cancer Res 2016;5:288-300. [Crossref] [PubMed]
- Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. [Crossref] [PubMed]
- Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med 1998;17:2815-34. [Crossref] [PubMed]
- Fossella FV, DeVore R, Kerr RN, et al. Randomized phase III trial of docetaxel versus vinorelbine or ifosfamide in patients with advanced non-small-cell lung cancer previously treated with platinum-containing chemotherapy regimens. The TAX 320 Non-Small Cell Lung Cancer Study Group. J Clin Oncol 2000;18:2354-62. [Crossref] [PubMed]
- Lara PN Jr, Redman MW, Kelly K, et al. Disease control rate at 8 weeks predicts clinical benefit in advanced non-small-cell lung cancer: results from Southwest Oncology Group randomized trials. J Clin Oncol 2008;26:463-7. [Crossref] [PubMed]
- De Gruttola VG, Clax P, DeMets DL, et al. Considerations in the evaluation of surrogate endpoints in clinical trials: Summary of a National Institutes of Health Workshop. Control Clin Trials 2001;22:485-502. [Crossref] [PubMed]
- Fleming TR, Powers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med 2012;31:2973-84. [Crossref] [PubMed]
- Lesko LJ, Atkinson AJ. Use of biomarkers and surrogate endpoints in drug development and regulatory decision making: Criteria, validation, strategies. Annu Rev Pharmacol Toxicol 2001;41:347-66. [Crossref] [PubMed]
- Fehrenbacher L, Spira A, Ballinger M, et al. Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial. Lancet 2016;387:1837-46. [Crossref] [PubMed]
- Gandara DR, von Pawel J, Mazieres J, et al. Atezolizumab Treatment Beyond Progression in Advanced NSCLC: Results From the Randomized, Phase III OAK Study. J Thorac Oncol 2018;13:1906-18. [Crossref] [PubMed]
- Lara PN Jr, Moon J, Redman MW, et al. Disease Control Rate at 8 Weeks Predicts Subsequent Survival in Platinum-Treated Extensive Stage Small-Cell Lung Cancer: Results From the Southwest Oncology Group (SWOG) Database. Clin Lung Cancer 2016;17:113-118.e111-2.
- Chibaudel B, Maindrault-Goebel F, Bachet JB, et al. PEPCOL: a GERCOR randomized phase II study of nanoliposomal irinotecan PEP02 (MM-398) or irinotecan with leucovorin/5-fluorouracil as second-line therapy in metastatic colorectal cancer. Cancer Med 2016;5:676-83. [Crossref] [PubMed]
- Kuramochi H, Ando M, Itabashi M, et al. Phase II study of bevacizumab and irinotecan as second-line therapy for patients with metastatic colorectal cancer previously treated with fluoropyrimidines, oxaliplatin, and bevacizumab. Cancer Chemother Pharmacol 2017;79:579-85. [Crossref] [PubMed]