Self-Regulatory Organizations; The Nasdaq Stock Market LLC; Notice of Filing of Amendment No. 2 and Order Granting Accelerated Approval of a Proposed Rule Change, as Modified by Amendment No. 2, To Amend Rules 4702(b)(14) and (b)(15) Concerning Dynamic M-ELO Holding Period, 62850-62863 [2023-19728]

Download as PDF 62850 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices relevant context.333 In addition, FINRA stated that after a panel dismisses a case at the conclusion of the case-in-chief, the firm must file an amended Uniform Application for Securities Industry Registration or Transfer (‘‘Form U4’’) for the associated person to report the final disposition of the case as dismissed.334 FINRA stated that along with the final disposition, an associated person can provide a brief summary or add context on Form U4 regarding the circumstances leading to the customer arbitration, as well as the current status or final disposition.335 This updated information is subsequently disclosed on the associated person’s BrokerCheck report, which is publicly available to investors.336 The Commission believes that this proposed rule change should promote transparency about FINRA’s arbitration process and help ensure consistent treatment of awards. Specifically, the proposed rule change equally requires all arbitration awards, including awards granting a motion to dismiss all claims, to be published. These published awards should provide current and future parties to an arbitration with data that could help inform the administration of their cases. The Commission acknowledges the commenter’s concern that a published award granting a motion to dismiss all claims may not reflect any defense raised by respondents. However, these concerns should be ameliorated by the fact that the Codes permit arbitrators to include a rationale underlying the award, providing relevant context to the dismissal of the claim such as the circumstances under which the claim was dismissed. In addition, an associated person may provide context on Form U4 regarding the circumstances leading to the customer arbitration, as ddrumheller on DSK120RN23PROD with NOTICES1 333 See id.; see also FINRA Rules 12904(f) and 13904(f). 334 See FINRA August Letter at 7 (citing FINRA By-Laws, Article V, Sections 2(c), 3(a) and 3(b)). 335 See id. at 7 n.30. 336 FINRA Rule 8312 (FINRA BrokerCheck Disclosure) governs the information FINRA releases to the public through its BrokerCheck system. Information available to investors through BrokerCheck includes, among other things, information reported on the most recently filed ‘‘Registration Forms’’ (with limited exceptions) for both member firms and registered individuals, and summary information about certain arbitration awards against the firm involving a securities or commodities dispute with a public customer; see also FINRA Rule 8312(b)(2)(A) (using the term ‘‘Registration Forms’’ to refer collectively to Form U4, the Uniform Termination Notice for Securities Industry Registration (Form U5), the Uniform Disciplinary Action Reporting Form (Form U6), the Uniform Application for Broker-Dealer Registration (Form BD), and the Uniform Request for BrokerDealer Withdrawal (Form BDW)). The BrokerCheck website is available at brokercheck.finra.org. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 well as the claim’s current status or final disposition. For these reasons, the Commission finds that this proposed rule change is reasonably designed to protect investors and the public interest. IV. Conclusion For the reasons set forth above, the Commission finds that the proposed rule change, as modified by Amendment No. 1, is consistent with Section 15A(b)(6) of the Exchange Act, which requires, among other things, that FINRA rules be designed to prevent fraudulent and manipulative acts and practices, promote just and equitable principles of trade, and, in general, protect investors and the public interest.337 It is therefore ordered pursuant to Section 19(b)(2) of the Exchange Act 338 that the proposed rule change (SR– FINRA–2022–033), as amended by Amendment No. 1, be, and hereby is, approved. For the Commission, by the Division of Trading and Markets, pursuant to delegated authority.339 Sherry R. Haywood, Assistant Secretary. [FR Doc. 2023–19729 Filed 9–12–23; 8:45 am] BILLING CODE 8011–01–P SECURITIES AND EXCHANGE COMMISSION [Release No. 34–98321; File No. SR– NASDAQ–2022–079] Self-Regulatory Organizations; The Nasdaq Stock Market LLC; Notice of Filing of Amendment No. 2 and Order Granting Accelerated Approval of a Proposed Rule Change, as Modified by Amendment No. 2, To Amend Rules 4702(b)(14) and (b)(15) Concerning Dynamic M–ELO Holding Period September 7, 2023. I. Introduction On December 21, 2022, The Nasdaq Stock Market LLC (‘‘Nasdaq’’ or ‘‘Exchange’’) filed with the Securities and Exchange Commission (‘‘Commission’’), pursuant to Section 19(b)(1) of the Securities Exchange Act of 1934 (‘‘Act’’) 1 and Rule 19b–4 thereunder,2 a proposed rule change to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with 337 15 U.S.C. 78o–3(b)(6). U.S.C. 78s(b)(2). 339 17 CFR 200.30–3(a)(12). 1 15 U.S.C. 78s(b)(1). 2 17 CFR 240.19b–4. 338 15 PO 00000 Frm 00081 Fmt 4703 Sfmt 4703 dynamic holding periods. The proposed rule change was published for comment in the Federal Register on January 10, 2023.3 On February 22, 2023, pursuant to Section 19(b)(2) of the Act,4 the Commission designated a longer period within which to approve the proposed rule change, disapprove the proposed rule change, or institute proceedings to determine whether to disapprove the proposed rule change.5 On March 9, 2023, the Exchange filed Amendment No. 1 to the proposed rule change, which amended and superseded the proposed rule change as originally filed. On April 7, 2023, the Commission provided notice of filing of Amendment No. 1 and instituted proceedings to determine whether to approve or disapprove the proposed rule change, as modified by Amendment No. 1.6 On July 6, 2023, pursuant to Section 19(b)(2) of the Act,7 the Commission designated a longer period on proceedings to determine whether to approve or disapprove the proposed rule change.8 On July 18, 2023, the Exchange filed Amendment No. 2 to the proposed rule change, which amended and superseded the proposed rule change as amended by Amendment No. 1. The Commission received comments on the proposed rule change.9 The Commission is publishing this Notice and Order to solicit comment on Amendment No. 2 in Sections II and III below, which sections are being published verbatim as filed by the Exchange, and to approve the proposed rule change, as modified by Amendment No. 2, on an accelerated basis. II. Self-Regulatory Organization’s Statement of the Terms of Substance of the Proposed Rule Change The Exchange proposes to amend Rules 4702(b)(14) and (b)(15) of the Exchange’s Rulebook to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with dynamic holding periods. This Amendment No. 2 3 See Securities Exchange Act Release No. 92844 (January 4, 2023), 88 FR 1438. 4 15 U.S.C. 78s(b)(2). 5 See Securities Exchange Act Release No. 96963, 88 FR 12710 (February 28, 2023). 6 See Securities Exchange Act Release No. 97263, 88 FR 22498 (April 13, 2023). 7 15 U.S.C. 78s(b)(2). 8 See Securities Exchange Act Release No. 97844, 88 FR 44423 (July 12, 2023). 9 All comments received by the Commission on the proposed rule change are available on the Commission’s website at: https://www.sec.gov/ comments/sr-nasdaq-2022-079/srnasdaq2022079. htm. E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices Background In 2018, the Exchange introduced the M–ELO, which is a Non-Displayed Order priced at the Midpoint between the National Best Bid and Offer (‘‘NBBO’’) and which is eligible for execution only against other eligible M– ELOs and only after a minimum of onehalf second passes from the time that the System accepts the order (the ‘‘Holding Period’’).11 In 2019, the Exchange introduced the M–ELO+CB, which closely resembles the M–ELO, except that a M–ELO+CB may execute at the midpoint of the NBBO, not only against other eligible M–ELOs (and M– ELO+CBs), but also against NonDisplayed Orders with Midpoint Pegging and Midpoint Peg Post-Only Orders (‘‘Midpoint Orders’’) that rest on the Continuous Book for at least onehalf second and have Trade Now enabled.12 When the Exchange designed M–ELO, it originally set the length of the Holding Period at one-half second because it determined that this time period would be sufficient to ensure that likeminded investors would interact only with each other, and with minimal market impacts. The Exchange believed that the longer length of the M– ELO Holding Period and its simplicity in design would provide greater protection for participants than they could achieve through competing delay mechanisms. In 2020, however, the Exchange shortened the length of the Holding Period to 10 milliseconds.13 The Exchange did so after studying two years of actual use and performance of M–ELOs, as well as customer feedback. That is, the Exchange came to understand that, while users of M–ELO and M–ELO+CB are less concerned with achieving rapid executions of their Orders than are other participants, they are not indifferent about the length of time in which their M–ELOs and M– ELO+CBs must wait before they are eligible for execution. Indeed, participants informed the Exchange that in certain circumstances, such as when they sought to trade symbols that on average had a lower time-to-execution than a half-second, they were reticent to enter M–ELOs or M–ELO+CBs. They indicated that the associated Holding Periods for these Order Types were longer than necessary to achieve the desired protections and that, during the residual portion of the Holding Periods, they risked losing out on favorable execution opportunities that would otherwise be available to them had they placed a non-MELO order. Based upon this feedback, the Exchange studied the potential effects of reducing the length of the Holding Periods for both M–ELOs and M– ELO+CBs (as well as for Midpoint Orders that would execute against M– ELO+CBs). Ultimately, the Exchange determined that it could reduce the Holding Periods to 10 milliseconds without compromising the protective power that M–ELO and M–ELO+CB are intended to provide to participants and 10 See SR–Nasdaq–2022–079 Amendment No. 1 (March 9, 2023), at https://www.sec.gov/comments/ sr-nasdaq2022-079/srnasdaq2022079-20159016327215.pdf. 11 See Securities Exchange Act Release No. 34– 82825 (March 7, 2018), 83 FR 10937 (March 13, 2018) (SR–NASDAQ–2017–074) (‘‘M–ELO Approval Order’’). 12 See Securities Exchange Act Release No. 34– 86938 (September 11, 2019), 84 FR 48978 (September 17, 2019) (SR–NASDAQ–2019–048) (‘‘M–ELO+CB Approval Order’’). 13 See Securities Exchange Act Release No. 34– 88743 (April 24, 2020), 85 FR 24068 (April 30, 2020) (SR–NASDAQ–2020–011) (‘‘M–ELO Timer Approval Order’’). supersedes the original filing and Amendment No. 1 10 in their entireties. The text of the proposed rule change is available on the Exchange’s website at https://listingcenter.nasdaq.com/ rulebook/nasdaq/rules, at the principal office of the Exchange, and at the Commission’s Public Reference Room. III. Self-Regulatory Organization’s Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change In its filing with the Commission, the Exchange included statements concerning the purpose of and basis for the proposed rule change and discussed any comments it received on the proposed rule change. The text of these statements may be examined at the places specified in Item IV below. The Exchange has prepared summaries, set forth in sections A, B, and C below, of the most significant aspects of such statements. A. Self-Regulatory Organization’s Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change ddrumheller on DSK120RN23PROD with NOTICES1 1. Purpose The Exchange proposes to amend Rules 4702(b)(14) and (15) of the Exchange’s Rulebook to replace the static 10 millisecond holding period requirements for its Midpoint Extended Life Order (‘‘M–ELO’’) and Midpoint Extended Life Order Plus Continuous Book (‘‘M–ELO+CB’’) Order Types with dynamic holding periods (‘‘Dynamic M– ELO and M–ELO+CB’’ or collectively, ‘‘Dynamic M–ELO’’). VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 PO 00000 Frm 00082 Fmt 4703 Sfmt 4703 62851 investors.14 Thus, the Exchange determined that shortening the Holding Periods to 10 milliseconds for M–ELOs and M–ELO+CBs would increase the efficacy of the mechanism while not undermining the power of those Order Types to fulfill their underlying purpose of minimizing market impacts. At the same time, the Exchange determined that a reduction in the Holding Periods to 10 milliseconds would dramatically add to the circumstances in which M– ELOs and M–ELO+CBs would be useful to participants. In its M–ELO Timer Approval Order, the Commission agreed with the Exchange: The Commission notes that, with the proposed ten-millisecond Holding Period and Resting Period, M–ELOs and M–ELO+CBs would continue to be optional order types that are available to investors with longer investment time horizons, including institutional investors. The Commission also believes that the proposal could make M–ELOs and M–ELO+CBs more attractive for securities that on average have a timeto-execution of less than one-half second and, for investors who currently do not use M–ELOs and M–ELO+CBs for these securities, provide optional order types that could enhance their ability to participate effectively on the Exchange. The Commission notes that, if market participants determine that the proposal would make M–ELOs and M– ELO+CBs less attractive for their particular investment objectives, such market participants may elect to reduce or eliminate their use of these optional order types. Moreover, as noted above, the Exchange will continue to conduct real-time surveillance to monitor the use of M–ELOs and M–ELO+CBs to ensure that such usage remains appropriately tied to the intent of the order types. If, as a result of such surveillance, the Exchange determines that the shortened 14 The Exchange examined each of its historical M–ELO executions to determine at what Midpoints of the NBBO the M–ELOs would have executed if their Holding Periods had been shorter than onehalf second (500 milliseconds). After examining the historical effects of shorter Holding Periods of between 10 milliseconds and 400 milliseconds, the Exchange determined that a reduction of the M– ELO Holding Period to as short as 10 milliseconds would have caused an average impact on mark-outs of only 0.10 basis points (across all symbols). In other words, compared to the execution price of an average M–ELO with a one-half second Holding Period, the Exchange found that a M–ELO with a 10 millisecond Holding Period would have had an average post-execution impact that was only a tenth of a basis point per share—a difference in protective effect that is immaterial. See Nasdaq, ‘‘The Midpoint Extended Life Order (M–ELO); M–ELO Holding Period,’’ available at https:// www.nasdaq.com/articles/the-midpoint-extendedlife-order-m-elo%3A-m-elo-holding-period-2020-0213 (analyzing effects of shortened Holding Periods on M–ELO performance). E:\FR\FM\13SEN1.SGM 13SEN1 62852 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices Holding Period does not serve its intended purpose or adversely impacts market quality, the Exchange would seek to make further recalibrations.15 For similar reasons and with even better potential results for participants, the Exchange now proposes to further refine the length of the Holding Periods for M–ELOs and M–ELO+CBs, this time through the application of innovative and patent pending machine learning technology. ddrumheller on DSK120RN23PROD with NOTICES1 Dynamic M–ELO After receiving feedback from participants that even 10 millisecond Holding Periods for M–ELO and M– ELO+CB may, at times, exceed what is necessary to accomplish the underlying intent of these Order Types, the Exchange began to experiment with making further refinements to the duration of the Holding Periods. Ultimately, the Exchange concluded that shorter Holding Periods could achieve the same, if not better results for participants in terms of mark-outs, but not in all circumstances. That is, where prices of the underlying securities are stable, and not subject to imminent unfavorable changes, M–ELOs and M– ELO+CBs face lower risks of confronting spread-crossing orders, such that shorter Holding Periods could suffice to protect M–ELOs and M–ELO+CB from such orders. In periods of heightened price volatility, however, M–ELOs and M– ELO+CBs also face heightened risks, such that longer Holding Periods would continue to be beneficial in protecting M–ELOs and M–ELO+CBs from such risks. Thus, the Exchange determined that another across-the-board reduction of the static 10 millisecond Holding Periods would be sub-optimal because it could impact the performance of the M– ELO and M–ELO+CB Order Types during periods of heightened volatility. In light of these observations, the Exchange tasked its artificial intelligence and machine learning laboratory (the ‘‘AI Core Development Group’’) to explore whether it could employ these innovative technologies to optimize the length of M–ELO and M– ELO+CB Holding Periods during various states of price volatility, and then to vary the lengths of the Holding Periods dynamically during the lifecycles of M– ELOs and M–ELO+CBs, with the objectives of improving the performance of these Order Types while also further reducing opportunity costs. 15 M–ELO Timer Approval Order, supra, at 85 FR 24069. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 As the Exchange explains in greater depth in the attached White Paper,16 the AI Core Development Group proceeded to develop an artificial intelligencebased timer control system that will achieve these objectives.17 The AI Core Development Group did so by using reinforcement learning techniques— machine learning paradigms which develop optimal solutions to problems over time by taking actions to solve them, generating feedback on the results of such actions, applying that feedback to direct and improve the next round of solutions, and then repeating the feedback loop until the paradigm achieves optimized solutions. In this instance, the AI Core Development Group applied reinforcement learning techniques to a simulation of the M–ELO Book that it constructed using a representative data set from the first quarter of 2022 (the ‘‘Training Period’’). The Training Period data consisted of 380 out of the 6,257 symbols on the M–ELO Book (accounting for approximately 67 percent of M–ELO volume). The symbols chosen reflect both activelytraded and thinly-traded securities, and both low-priced and high-priced securities. The AI Core Development Group then developed a machine learning model and applied it to the Training Period data. The Group programmed the model to value the achievement of higher fill rates or lower mark-outs than that which occurred in a historical simulation of M–ELOs and M–ELO+CBs involving the Training Period data.18 The Group then programmed the model to seek to achieve its goals by taking one of five possible actions with respect to the duration of the Holding Periods at 16 See Diana Kafkes et al., ‘‘Applying Artificial Intelligence & Reinforcement Learning Methods Towards Improving Execution Outcomes,’’ SSRN, October 19, 2022, available at https:// papers.ssrn.com/sol3/papers.cfm?abstract_ id=4243985 (attached hereto [sic] without modification from the prior version as Exhibit 3(a)) (the ‘‘White Paper’’). 17 Although the AI Core Development Group acknowledges that an optimal Holding Period would update with every incoming order, it determined that training a reinforcement learning model on every order would be too difficult to program and too difficult to implement given the nanosecond latency requirements of the Exchange. The Group then investigated more feasible update cadences and determined the point at which optimal outcomes were best balanced with the level of programming and implementation difficulty to be between 15 and 30 second updates. Ultimately, the Group chose a 30 second update cadence to give the model the greatest opportunity to learn between potential actions. 18 As the White Paper explains, the Group developed a model to simulate activity on the Exchange involving M–ELOs and M–ELO+CBs during the Training Period. See White Paper, supra, at 10. PO 00000 Frm 00083 Fmt 4703 Sfmt 4703 30 second intervals 19 for each symbol during each trading day of the Training Period. That is, at each 30 second internal, the model evaluated market conditions for each symbol over the prior 30 second period and either kept the Holding Periods the same, increased/decreased them by 0.25 milliseconds, or increased/decreased them by 0.50 milliseconds.20 After each decision-making round, the model utilized the results to inform its actions at the next 30 second increment. In making its decisions, the model (again, drawing upon a combination of historical SIP and M–ELO-specific data) considered 142 categories of data points.21 A confluence of data points that correlated with an increase in volatility tended to cause the model to increase the durations of Holding Periods, including increases in the standard deviation of NBBO prices, the number of unique participants placing sell orders on M–ELO and M–ELO+CB, and the volume-weighted average of the NBBO spread. Conversely, a confluence of data points that correlated with greater price stability tended to cause the model to decrease the durations of Holding periods, such as an increase in the median and max number of shares per trade and the number of resting bids left in the M–ELO and M–ELO+CB Book. The AI Core Development Team produced variations of its model that prioritized achievement of the lowest mark-outs, the highest fill rates, and a blend of these two objectives.22 Through 19 See id. AI Core Development Group experimented with a range of permissible Holding Period durations. Ultimately, it concluded that it could produce better outcomes for M–ELO and M– ELO+CB participants than the existing approach using Holding Periods as low as 0.25 milliseconds and as high as 2.5 milliseconds, under normal market conditions. 21 Nasdaq attaches a full list of these data elements (attached hereto [sic] as ‘‘Exhibit 3(b))’’, along with an observation of the strength of the correlations that currently exist between changes to those data values and decisions the system makes to set the duration of Holding Periods at any given time. The Exchange notes that the version of this list attached to this Amendment No. 2 supersedes prior versions attached to prior versions of this filing. This version of the list includes expanded explanations of the terminology used therein. See also White Paper, supra, at 31, for a description of these features. 22 The AI Core Development Group also applied to the model a paradigm called ‘‘retraining’’ to combat the degradation of model performance that can otherwise occur as the reference data it uses for initial comparison becomes stale. Finally, the AI Core Development group added a stability protection mechanism to the model to provide maximum production to participants in the event that the model observes extraordinary levels of instability in the National Best Bid and Offer during the prior three seconds as compared to reference data. When the model detects such instability, it is 20 The E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices ddrumheller on DSK120RN23PROD with NOTICES1 a process of learning and experimentation involving a combination of historical and simulated data, the AI Core Development Group settled on a Dynamic M–ELO model that achieved substantial simulated performance improvements for users of M–ELO and M–ELO+CB—both in terms of mark-outs and fill rates—as compared to the static 10 millisecond Holding Periods. As the White Paper explains in greater detail, Dynamic M–ELO yielded an average combined volume-weighted (simulated) improvement of 31.7 percent, including a 20.3 percent increase in fill rates and a 11.4 percent reduction in mark-outs.23 The White Paper provides a more fulsome explanation of these improvements.24 Based upon these exciting results, the Exchange now proposes to amend Rule 4702(b)(14) and (15) to replace the static 10 millisecond timers applicable to M– ELO and M–ELO+CB with Dynamic M– ELO Holding Periods. Using the Exchange’s ‘‘proprietary assessment of market conditions’’ 25 and patent pending technology, the Dynamic M– ELO system will evaluate and, as it deems necessary, adjust the length of the Holding Periods for each symbol comprising M–ELOs and M–ELO+CBs (and Midpoint Orders on the Continuous Book that opt to interact with M–ELO+CBs after resting on the Book) every 30 seconds throughout the Market Hours (each such 30 second interval, a ‘‘Change Event’’). In so doing, Dynamic M–ELO will help participants to achieve a more optimized blend of the underlying purposes of the M–ELO and M–ELO+CB Order Types: protection against adverse selection programmed to increase the length of the Holding Period to 12 milliseconds for a period of 750 milliseconds. 23 See White Paper, supra, at 22. 24 See id. 25 As set forth in the proposed rule text, the phrase ‘‘proprietary assessment of market conditions’’ refers to the Exchange’s evaluation of prevailing market conditions for a given symbol using an algorithm programmed to set a Holding Period duration which, at each Change Event, achieves an optimal blend of two objectives: maximization of M–ELO fill rates; and minimization of M–ELO mark-out rates. As the rule text states and as is discussed below, the algorithm ingests and analyzes 142 data points, which the Exchange identifies and describes in Exhibit 3b hereto. The Exchange derives these data from a combination of public data and M–ELO data feeds. Furthermore, the Exchange conducts weekly retrainings of the algorithm, outside of Market Hours, to improve its performance relative to the immediately preceding period (in terms of the two aforementioned objectives). The Exchange deploys a retrained version of the algorithm only if it determines that doing so will, in fact, improve its performance relative to the immediately preceding period. The Exchange provides further information about the algorithm and the retraining process in a White Paper attached hereto [sic] as Exhibit 3a. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 (low mark-outs) without sacrificing opportunities to achieve high-quality executions (high fill rates). A proposed M–ELO or M–ELO+CB with a Dynamic Holding Period will operate as follows. At the outset of Market Hours (approximately 9:30:00 a.m.), the Exchange will impose initial Holding Periods of 1.25 milliseconds for M–ELOs and M–ELO+CBs in all symbols. Thereafter, Holding Periods for a given symbol will become eligible to change dynamically from the initial duration beginning at 9:30:30 a.m. and then at 30 second intervals thereafter during Market Hours. The Exchange will then apply to the M–ELO or M– ELO+CB Order a Holding Period that is of the duration that prevailed at the time of entry. For example, if participant A enters a M–ELO for symbol XYZ at 9:30:25 a.m., then Holding Period for that M–ELO will be 1.25 milliseconds. If at 9:30:30:00 a.m., the System decides to lower the duration of the Holding Period by 0.50 milliseconds, and then participant B enters a M–ELO for symbol XYZ at 9:30:45 a.m., then the System will assign a 0.75 millisecond Holding Period to participant B’s M– ELO. To be clear, the System will determine Dynamic M–ELO Holding Periods independently for M–ELOs and M–ELO+CBs in each symbol. During normal market conditions, the range of potential Holding Period durations for M–ELOs and M–ELO+CBs will be between 0.25–2.50 milliseconds, with the Holding Period duration being eligible to change by increments of either 0.25 or 0.50 milliseconds at each Change Event. Thus, if the Holding Period for a M–ELO in symbol XYZ is set at 0.75 milliseconds at 2:22:11 p.m., and at 2:22:41 p.m., the System determines to increase the duration of the Holding Period, it may do so only by 0.25 or 0.50 milliseconds during that event. When a Change Event occurs, and the System determines to adjust the duration of a Holding Period for a symbol, that adjustment will apply, not only to all M–ELOs and M–ELO+CBs for that symbol entered within the 30 second period after the Change Event occurs, but also to M–ELOs and M– ELO+CBs entered prior to the Change Event with unexpired Holding Periods (with applicability retroactive to the time of Order acceptance). Thus, if a participant enters a M–ELO in symbol XYZ at 1:14:299 p.m., and the prevailing Holding Period applicable to that M– ELO is 2 milliseconds, and at 1:14:30 p.m., the System modifies the Holding Period to be 1.5 milliseconds, then the M–ELO will become eligible to execute at 1:14:3005 p.m. This is the case PO 00000 Frm 00084 Fmt 4703 Sfmt 4703 62853 because the M–ELO will have already expended 1 millisecond of its Holding Period as of the time of the Change Event; thereafter, the M–ELO will need to rest only another 0.5 milliseconds to become eligible to execute under the new 1.5 millisecond Holding Period (as measured from 1:14:299 p.m.). This last feature ensures that the M–ELO Book maintains time priority among M–ELOs and M–ELO+CBs in a dynamic environment. That is, it ensures that no M–ELO or M–ELO+CB with an unexpired Holding Period at the time of a Change Event will end up becoming eligible to execute later than a M–ELO entered after the Change Event which has a shorter Holding Period applicable to it. If at any time, the System detects extraordinary instability in a symbol, then the System will activate a ‘‘stability protection mechanism’’ to provide an extra layer of protection to M–ELO and M–ELO users from the heightened risks of adverse selection that exists during such periods of instability.26 The stability protection mechanism will override the prevailing Holding Periods for M–ELOs and M–ELO+CBs in a symbol experiencing extraordinary instability and immediately increase the duration of those Holding Periods to 12 milliseconds for a period of 750 milliseconds. The System may activate the stability protection mechanism even between Change Events. The System will evaluate, at each NBBO update, whether market conditions remain extraordinarily unstable and, if so, it will restart the 750 millisecond Stability Protected Period and maintain the 12 millisecond Holding Period until conditions stabilize. Once the System determines that market conditions have stabilized (i.e., all measurements for the symbol are at or below the threshold value throughout the duration of the prevailing Stability Protected Period), the System will revert the duration of the Holding Periods to that which prevailed as of the Change Event that 26 For purposes of this Rule, the System determines that ‘‘extraordinary instability’’ for a symbol exists through observations it makes following every change in the NBBO for that symbol that occurs during the trading day. When the NBBO changes, the System looks back at the prior three seconds of trading and measures the difference between the highest and the lowest NBBO midpoint values that occurred during that period, and then it compares that measurement to a threshold value for the symbol. The System concludes that extraordinary instability exists for a symbol if the measurement exceeds the threshold value. The threshold value for a symbol, in turn, is the difference between the highest and the lowest NBBO midpoint values for the symbol that, if applied to its trading activity during the prior trading day, would have caused the System to deem trading in the symbol to be extraordinarily unstable for as close to one percent of that day as possible. E:\FR\FM\13SEN1.SGM 13SEN1 ddrumheller on DSK120RN23PROD with NOTICES1 62854 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices occurred immediately prior to the activation of the stability protection mechanism or, if the stability protection mechanism was active when a Change Event occurred, to the duration selected at the immediately preceding Change Event. The System will then proceed to reevaluate the duration of the Holding Periods as per the regular schedule of Change Events. The following is an illustration of the operation of the stability protection mechanism. At 11:10:04 a.m., the prevailing Holding Period for M–ELOs in symbol XYZ is 1.5 milliseconds. At the same time, the NBBO for symbol XYZ updates. The System looks back at the prior three seconds of trading in symbol XYZ and finds that during that period, the highest observed NBBO midpoint was $10.05, and the lowest was $10.00, such that the difference between these two values is a range of $0.05. The System then looks back at trading behavior for symbol XYZ during the immediately preceding trading day. In doing so, the System calculates the value of the threshold that would have caused the symbol to be deemed extraordinarily unstable for one percent of the trading day; the System determines that this threshold value is a range of $0.03. The System then compares the $0.03 threshold to its measurement of the prior three seconds of NBBO changes ($0.05), and concludes that over these past three seconds, the symbol is extraordinarily unstable. Accordingly, the System activates the stability protection mechanism and the Holding Period for M–ELOs in symbol XYZ immediately increases to 12 milliseconds for a period of 750 milliseconds. However, 5 milliseconds after the Stability Protection Period commences, the NBBO updates again, thus prompting the System to repeat its assessment of the stability of the symbol in light of the update. This reassessment reveals that the symbol remains unstable, such that a new Stability Protection Period of 750 milliseconds begins at that time (overriding the preexisting Period). Over the course of this new Stability Protection Period, the NBBO shifts two more times, but each of the ensuing reassessments indicate that the NBBO ranges for the symbol have fallen below the $0.03 threshold. The Stability Protection Period elapses 750 milliseconds after it began with the symbol remaining stable. Thus, the Holding Period reverts to 1.5 milliseconds. If the Exchange halts trading in a symbol, then upon resumption of trading, any new M–ELO or M–ELO+CB in that symbol and any pending M–ELO or M–ELO+CB in that symbol with an VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 unexpired Holding Period will be subject to a new 12 milliseconds Holding Period (running from the time when trading resumes) until the next scheduled Change Event, at which point the System may determine to adjust that Holding Period to a duration within the range applicable under normal market conditions.27 If, however, the System determines that extraordinary instability in the symbol exists, it will instead determine to activate the stability protection mechanism and maintain the duration of the Holding Period at 12 milliseconds for another 750 milliseconds. This design will help to ensure that M–ELOs and M–ELO+CBs receive added protection coming out of halt conditions.28 The Exchange notes that same dynamic process described above will also apply to and govern the time periods during which Midpoint Orders on the Continuous Book must rest before they will become eligible to interact with M–ELO+CBs (provided that participants have opted for their Midpoint Orders to interact with M– ELO+CBs). Thus, the same Holding Period duration that the System sets for a M–ELO+CB in a symbol during Regular Market Hours will also be the length of time that a Midpoint Order must rest on the Continuous Book must rest before it may interact with a M– ELO+CB. Apart from these impacts of Dynamic Holding Periods, M–ELOs and M– ELO+CBs will continue to behave as they do now in all respects, and as set forth in Rules 4702(b)(14) and (15). It is important to note that within the parameters discussed herein and in the White Paper, the Exchange will continue to re-train Dynamic M–ELO and M–ELO+CB on a weekly basis (outside of market hours) so that the model will continue to learn from and act upon the basis of more recent SIP and M–ELO book data sets, and further 27 Prior to commencement of a new 12 millisecond Holding Period for a new or pending M–ELO or M–ELO+CB following a Halt, the System will first determine whether the M–ELO or M– ELO+CB is or remains eligible for execution. That is, the Holding Period will commence only if, upon commencement of trading following the Halt, the midpoint price for the Order is within the limit set by the participant. If not, the System will hold the Order until the midpoint falls within the limit set by the participant, at which time the 12 millisecond Holding Period will commence. 28 Also as a safeguard, the System will apply a default Holding Period of 12 milliseconds to a M– ELO or M–ELO+CB if ever it fails to receive a signal during a Change Event as to whether the System should adjust or maintain the duration of the prevailing Holding Period. The System will continue to apply the default 12 millisecond Holding Period until the next Change Event where the signal is restored and the System is able to act dynamically again. PO 00000 Frm 00085 Fmt 4703 Sfmt 4703 improve its performance over time. The retraining process should not result in dramatic or unpredictable changes to the behavior of Dynamic M–ELO. The retraining process will not retrain the model from scratch each week; rather, it will retain the model’s existing data inputs, knowledge base, and objectives—all without alteration. Retraining will result in new behaviors only as needed to address new scenarios that the model did not confront previously, and even then, only in a manner designed to further optimize outcomes, i.e., reduce mark-outs or increase fill rates. If the System assesses that a retrained model would be worse than the existing model in achieving its objectives, then the System will continue to use the existing model and discard the retrained model. This retraining process is a standard and accepted practice for use of deep learning models; it helps to ensure that deep learning models not only work well, but that they continue to work well in dynamic circumstances.29 The Exchange will not modify the underlying structure of Dynamic M– ELO and M–ELO+CB without first obtaining the Commission’s approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model my adjust the Holding Periods, the range of Holding Period durations available to M–ELOs and M– ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability.30 Although the Exchange will seek Commission approval prior to changing any of the data elements that the model considers, the Exchange will not seek Commission approval prior to retraining 29 During periods where the model is not undergoing retraining, the System will behave predictably from day to day, such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2. 30 In addition to the proposed changes described above, the Exchange proposes to delete an extraneous reference in Rule 4702(b)(15) to M– ELO+CB being eligible to execute against a Midpoint Order on the Continuous Book if the Continuous Book order has the ‘‘Midpoint’’ Trade Now Attribute enabled. In a prior filing, the Exchange folded the concept of ‘‘Midpoint Trade Now’’ into the general ‘‘Trade Now’’ Attribute. See Securities Exchange Act Release No. 34–92180 (June 15, 2021), 86 FR 33420 (June 24, 2021)(SR– NASDAQ–2021–044). E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices ddrumheller on DSK120RN23PROD with NOTICES1 the model to adjust the weighting it applies to those data elements. To aid investors in understanding and evaluating Dynamic M–ELO, Nasdaq will continue to publish weekly and monthly transparency statistics on Nasdaqtrader.com, as it does now, about the performance of its M–ELOs and M– ELO+CBs, including statistics listing the weekly numbers of shares and trades in M–ELOs by symbol, weekly aggregated M–ELO share and trade data, and monthly aggregated block data.31 Nasdaq also will continue to disclose monthly data on Nasdaq.com, as it does now (the M–ELO Monthly Report), about M–ELO and M–ELO+CB markouts (quote stability by time horizon) and fill rates.32 Moreover, Nasdaq will add statistics to the M–ELO Monthly Report about how frequently, on average, the System changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M–ELO and M–ELO+CB trading volumes. Nasdaq will retain copies of each historical iteration of its models as part of its books and records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time. Furthermore, Nasdaq will publish an equity trader alert in advance of deploying a retrained version of Dynamic M–ELO whenever Nasdaq has reason to anticipate that the retrained version will produce results that differ materially from the prior version, i.e., a projected change in mark-outs or fillrates of 10% or more in either direction. The Exchange acknowledges that systems necessary to implement Dynamic M–ELO, including the systems proposed that include model development and retraining processes, are ‘‘SCI Systems’’ within the meaning of Regulation Systems Compliance and Integrity (‘‘Reg. SCI’’),33 and that the 31 See https://www.nasdaqtrader.com/Trader. aspx?id=MELOSymbolData. 32 See, e.g., https://www.nasdaq.com/docs/MELO-Monthly-Report. Nasdaq understands that current users of M–ELO and M–ELO independently monitor the performance of these Order Types. Nasdaq often receives feedback from such users about M–ELO and M–ELO+CB performance, which Nasdaq then factors into decisions about improvements and enhancements. Nasdaq expects that this feedback loop will continue after implementation of Dynamic M–ELO. 33 17 CFR 242.1000 et seq. As set forth in Reg. SCI, the term ‘‘SCI Systems’’ means ‘‘means all computer, network, electronic, technical, automated, or similar systems of, or operated by or on behalf of, an SCI entity that, with respect to securities, directly support trading, clearance and settlement, order routing, market data, market regulation, or market surveillance.’’ Id. at 242.1000. An ‘‘SCI Entity’’ means ‘‘an SCI self-regulatory organization, SCI alternative trading system, plan VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 Exchange, as an SCI Entity, remains responsible for compliance with all requirements of Reg. SCI, including, without limitation, to have policies and procedures reasonably designed to ensure that its SCI Systems operate in a manner that complies with the Act and the rules and regulations thereunder and Exchange’s rules and governing documents, among them a plan for assessments of the functionality of SCI Systems designed to detect systems compliance issues, including by responsible SCI personnel and by personnel familiar with applicable provisions of the Act and the rules and regulations thereunder and Exchange’s rules and governing documents. Implementation The Exchange intends to make the proposed change effective for M–ELOs and M–ELO+CBs in the Second or Third Quarter of 2023, but that time frame is subject to change. The Exchange will publish a Trader Alert in advance of making the proposed change effective. 2. Statutory Basis The Exchange believes that its proposal is consistent with Section 6(b) of the Act,34 in general, and furthers the objectives of Section 6(b)(5) of the Act,35 in particular, in that it is designed to promote just and equitable principles of trade, to remove impediments to and perfect the mechanism of a free and open market and a national market system, and, in general to protect investors and the public interest, by allowing for more widespread use of M– ELOs and M–ELO+CBs. When the Commission approved the M–ELO and the M–ELO+CB, it determined that these Order Types are consistent with the Act because they ‘‘could create additional and more efficient trading opportunities on the Exchange for investors with longer investment time horizons, including institutional investors, and could provide these investors with an ability to limit the information leakage and the market impact that could result from their orders.’’ 36 Nothing about the Exchange’s proposal should cause the Commission to revisit or rethink this determination. Indeed, the proposal will not alter the fundamental design of these Order Types, the manner in which they operate, or their effects. processor, exempt clearing agency subject to ARP, or SCI competing consolidator.’’ Id. 34 15 U.S.C. 78f(b). 35 15 U.S.C. 78f(b)(5). 36 M–ELO Approval Order, supra 83 FR at 10938– 39; M–ELO+CB Approval Order, supra, 84 FR at 48980. PO 00000 Frm 00086 Fmt 4703 Sfmt 4703 62855 Even with Dynamic M–ELO Holding Periods, M–ELOs and M–ELO+CBs will continue to provide their users with protection against information leakage and adverse selection—and they will do so at levels which are substantially undiminished from that which they provide now.37 At the same time, however, the proposal will benefit market participants and investors by reducing the opportunity costs of utilizing M– ELOs and M–ELO+CBs. The proposal, in other words, will re-calibrate the lengths of the Holding Periods so that M–ELOs and M–ELO+CBs will operate in the ‘‘Goldilocks’’ zone—their Holding Periods will not be so short as to render them unable to provide meaningful protections against information leakage and adverse selection, but the Holding Periods also will not be too long so as to cause participants and investors to miss out on favorable execution opportunities. Nasdaq believes the proposal will render M–ELOs and M– ELO+CBs more useful and attractive to market participants and investors, and this increased utility and attractiveness, in turn, will spur an increase in M–ELO and M–ELO+CB use cases on the Exchange, both from new and existing users of M–ELOs and M–ELO+CBs. Ultimately, the proposal should enhance market quality by increasing opportunities for midpoint executions on the Exchange. As Nasdaq explained above, the Proposal will operate within strict, welldefined, and transparent parameters. Although it will undergo weekly retraining (outside of market hours),38 such retraining will aim to improve the performance of the model in achieving its twin objectives; retraining will not alter the inputs, objectives, or basic design parameters of Dynamic M–ELO without prior Commission approval.39 Moreover, the Exchange will not deploy a retrained model if it fails to achieve performance improvements. To aid 37 See note 6, supra. be clear, performance statistics for Dynamic M–ELO cited herein and in the White Paper are based upon data derived from weekly, not daily retrainings. 39 As discussed above, Nasdaq will not seek Commission approval prior to allowing the model, as part of its re-training process, to vary the weighting of the data elements it ingests. Nasdaq believes this is appropriate because such variance will only occur to the extent that it will improve the model’s performance with respect to predefined objectives. Nasdaq will alert traders if the retraining process would result in substantial performance changes, and it will also publish statistics to help participants to assess performance themselves. Moreover, Nasdaq will retain historical iterations of its models for the Commission’s review, should it wish to examine how these models have changed over time. 38 To E:\FR\FM\13SEN1.SGM 13SEN1 62856 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices ddrumheller on DSK120RN23PROD with NOTICES1 investors in evaluating Dynamic M– ELO, the Exchange will publish statistics about its performance, including as to mark-outs and fill rates, as well as statistics about how frequently the System changes Holding Period durations. To further facilitate accountability, the Exchange will retain each historical iteration of its model as part of its books and records, and make such information available to the Commission, upon request. The Exchange will also publish equity trader alerts whenever retraining will result in a performance change of 10% or more. Nasdaq notes that the twin objectives it prescribes for the model involve the absolute values of mark-outs and fill rates; they are not designed to further the performance of any participant or any category of participant. Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M–ELO, as compared to the ‘‘static’’ M–ELO, for those firms that accounted for more than 95% of M–ELO activity on the Exchange during Q1 2022.40 The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M–ELO versus static M–ELO, with the volume-weighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant (i.e., nature of firm; size of firm) benefitted from Dynamic M–ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination.41 The Exchange notes that use of Dynamic M–ELOs and M–ELO+CBs remains voluntary for all market participants. Accordingly, if any market participant feels that the dynamic 40 Beyond this grouping of participants, the activity levels of other individual M–ELO participants were so small as to be insignificant. In many cases, these participants entered only a handful of M–ELOs during the study period. As such, the Exchange believes it is reasonable to exclude such participants from its analysis to avoid their data distorting the results. 41 The Exchange will review its AI model periodically to affirm that it continues to perform in accordance with the Exchange’s rules and has not introduced any harmful bias in favor of or against any participant or category of participants. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 Holding Periods are still too long or too short or because competing venues offer more attractive delay mechanisms, then the participants are free to pursue other trading strategies or utilize other trading venues. They need not utilize Dynamic M–ELOs or M–ELO+CBs. Furthermore, the design of DynamicMELO would constitute an ‘‘established, non-discretionary’’ method that is consistent with the definition of an exchange, as set forth in SEC Rule 3b– 16.42 The Commission stated as follows when it adopted Rule 3b–16: A system uses established nondiscretionary methods either by providing a trading facility or by setting rules governing trading among subscribers. The Commission intends for ‘‘established, non-discretionary methods’’ to include any methods that dictate the terms of trading among the multiple buyers and sellers entering orders into the system. Such methods include those that set procedures or priorities under which open terms of a trade may be determined. For example, traditional exchanges’ rules of priority, parity, and precedence are ‘‘established non-discretionary methods,’’ as are the trading algorithms of electronic systems. Similarly, systems that determine the trading price at some designated future date on the basis of pre-established criteria (such as the weighted average trading price for the security on the specified date in a specified market or markets) are using established, non-discretionary methods.43 Nothing in the Reg. ATS Adopting Release or in any of its illustrative examples suggests that Dynamic M–ELO would constitute an exercise of discretionary behavior. Dynamic M– ELO will handle and execute Orders according to published, pre-determined rules that are disclosed to the public and which provide reasonable notice of how the Order Type will behave.44 To the extent that the design of the System 42 See 17 CFR 240.3b–16(a)(2) (‘‘(a) An organization, association, or group of persons shall be considered to constitute, maintain, or provide ‘a market place or facilities for bringing together purchasers and sellers of securities or for otherwise performing with respect to securities the functions commonly performed by a stock exchange,’ as those terms are used in section 3(a)(1) of the Act, (15 U.S.C. 78c(a)(1)), if such organization, association, or group of persons: (1) Brings together the orders for securities of multiple buyers and sellers; and (2) Uses established, non-discretionary methods (whether by providing a trading facility or by setting rules) under which such orders interact with each other, and the buyers and sellers entering such orders agree to the terms of a trade.’’). 43 See Securities Exchange Act Release No. 40760 (December 8, 1998), 63 FR 70844, 70850 (December 22, 1998). 44 See id. at 70900 (‘‘an essential indication of the non-discretionary status of rules and procedures is that those rules and procedures are communicated to the systems users’’ and ‘‘[t]hus, participants have an expectation regarding the manner of execution— that is, if an order is entered, it will be executed in accordance with those procedures and not at the discretion of a counterparty or intermediary.’’). PO 00000 Frm 00087 Fmt 4703 Sfmt 4703 permits variation in the Holding Periods for such Orders, it does so by design. The range of potential variations, the objectives that such variations are intended to achieve, and the factors that determine when such variations may occur are also predetermined and set forth in the Exchange’s Rules or otherwise disclosed to the public. The mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.45 Even when the System makes decisions about changing the Holding Periods, the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.46 45 Cf. id. at 70851 (explaining that a traditional block trading desk is an example of a system that does not use established, non-discretionary methods because the operators of such desks do not act according to fixed procedures known to their customers, but instead shop orders around for potential counterparties and make their own determinations as to whether and how to execute block orders, including by sometimes deciding to take a proprietary position in part of the block order). 46 See id. at 80755 (describing an example of a system that would be non-discretionary in nature: ‘‘System I permits participants to enter a range of ranked contingent buy and sell orders at which they are willing to trade securities. These orders are matched based on a mathematical algorithm whose priorities are designed to achieve the participants’ objectives. System I does not display orders to any participants. System I is included under Rule 3b– 16.’’); see also Securities Exchange Act Release No. 34–89686 (August 20, 2020), 85 FR 54438, at 54445, n.92 (September 1, 2020) (Order approving SR–IEX– 2019–15) (rejecting argument that IEX’s D-Limit order time is an exercise of discretion because ‘‘DLimit orders will not allow IEX to exercise any discretion on any particular order by deviating from the CQI and D-Limit functionality, which is hardcoded in the IEX rulebook.’’; Securities Exchange Act Release No. 34–78101 (June 17, 2016), 81 FR 41141, at 41153(June 17, 2016) (Order approving IEX Form 1 and D-Peg Order Type) (‘‘the Commission does not believe that the hardcoded conditionality of the IEX proposed ‘‘discretionary’’ peg order type provides IEX with actual discretion or the ability to exercise individualized judgment when executing an order. Rather, if IEX’s fixed formula determines the quote to be stable, the discretionary peg order can execute up to the midpoint; if it does not deem the quote to be stable, then it will hold the order to its pegged price. As such, IEX would not exercise discretion over the routing and execution of a resting order’’). Nasdaq does not believe that it is necessary to codify its mathematical formula for Dynamic M–ELO in its Rules because Nasdaq has disclosed sufficient information in its Rules and in its filing to inform the public as to the possible and expected behaviors associated with Dynamic M–ELO, as well as a means for the Commission and/or investors to verify whether Dynamic M–ELO is performing appropriately. Much as the Commission does not E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices ddrumheller on DSK120RN23PROD with NOTICES1 The Exchange notes that it will continue to conduct real-time surveillance to monitor the use of M– ELOs and M–ELO+CBs to ensure that such usage remains appropriately tied to the intent of the Order Types. If, as a result of such surveillance, the Exchange determines that the Dynamic M–ELO Holding Periods do not serve their intended purposes, or adversely impact market quality, then the Exchange will seek to make further recalibrations. Nasdaq does not believe that the design of Dynamic M–ELO lends itself to potential manipulation by a single participant or a small group of participants because the System makes determinations regarding Holding Periods based upon prevailing marketwide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M– ELOs involving that symbol. Manipulation of the System also would be difficult to accomplish given the large number of variables that factor into the System’s decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, the System may vary over time. Any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of affecting such manipulation. Nevertheless, the Exchange will surveil for indications of manipulation and act accordingly if it detects such indications. B. Self-Regulatory Organization’s Statement on Burden on Competition The Exchange does not believe that the proposed rule change will impose any burden on competition not necessary or appropriate in furtherance of the purposes of the Act. To the contrary, the Exchange believes that this proposal will promote the competitiveness of the Exchange by rendering its M–ELO and M–ELO+CB Order Types more attractive to participants. The Exchange adopted the M–ELO and M–ELO+CB as pro-competitive measures intended to increase participation on the Exchange by allowing certain market participants that may currently be underserved on require an exchange to codify the source code it uses to effectuate other behaviors or actions that it explains in its Rules, including the behaviors of other complex Order Types, there is no basis to require codification of the Dynamic M–ELO formula in this instance. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 regulated exchanges to compete based on elements other than speed. The proposed change continues to achieve this purpose. With Dynamic M–ELO Holding Periods, both M–ELOs and M– ELO+CBs will afford their users with a level of protection from information leakage and adverse selection that is better from what is achievable at present.47 At the same time, the Dynamic Holding Periods will increase opportunities to interact with other likeminded investors with longer time horizons while also lowering the opportunity costs for participants that utilize M–ELOs and M–ELO+CBs, particularly for securities that trade within the ‘‘Goldilocks’’ zone. In sum, the proposed changes will not burden competition, but instead may promote competition for liquidity in M–ELOs and M–ELO+CBs by broadening the circumstances in which market participants may find such Orders to be useful. With the proposed changes, market participants will be more likely to determine that the benefits of entering M–ELOs and M–ELO+CBs outweigh the risks of doing so. The proposed change will not place a burden on competition among market venues, as any market may adopt an order type that operates similarly to a M–ELO or a M–ELO+CB with Dynamic M–ELO Holding Periods. C. Self-Regulatory Organization’s Statement on Comments on the Proposed Rule Change Received From Members, Participants, or Others No written comments were either solicited or received. IV. Discussion and Commission Findings After careful review, the Commission finds that the Exchange’s proposal to adopt Dynamic M–ELO is consistent with the requirements of the Act and the rules and regulations thereunder applicable to a national securities exchange.48 In particular, the Commission finds that the proposed 47 See White Paper, supra. approving this proposed rule change, the Commission has considered the proposed rule’s impact on efficiency, competition, and capital formation. See 15 U.S.C. 78c(f). One commenter questioned whether Nasdaq’s pending patent applications for the systems it will use to operate Dynamic M–ELO imposes an unnecessary or inappropriate burden on competition. See Letter from R.T. Leuchtkafer, dated January 21, 2023 (‘‘Leuchtkafer Letter 1’’), at 3. The Commission does not believe that the sole fact that Nasdaq has a pending patent application for the technology it has developed to operate the Dynamic M–ELO is indicative that the operation of Dynamic M–ELO on the Exchange would place an inappropriate burden on competition. As explained below, Nasdaq has provided sufficient public disclosure and analysis to explain how Dynamic M–ELO will operate. 48 In PO 00000 Frm 00088 Fmt 4703 Sfmt 4703 62857 rule change, as modified by Amendment No. 2, is consistent with Sections 6(b)(5) and 6(b)(8) of the Act.49 Section 6(b)(5) of the Act requires that the rules of a national securities exchange be designed, among other things, to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers.50 Section 6(b)(8) of the Act requires that the rules of a national securities exchange not impose any burden on competition that is not necessary or appropriate in furtherance of the purposes of the Act.51 Nasdaq’s Obligation To Sufficiently Explain Its Proposed Rule Change The burden to demonstrate that a proposed rule change is consistent with the Act and rules and regulations thereunder is on the self-regulatory organization (‘‘SRO’’) proposing a rule change.52 Each proposed SRO rule change must be ‘‘accompanied by a concise general statement of the basis and purpose of such proposed rule change.’’ 53 As described in more detail below, several commenters argued that the proposal did not provide sufficient information with respect to the operation of Dynamic M–ELO, or that the information provided was not ‘‘clear and comprehensible,’’ as required by Form 19b–4. For the reasons articulated below, the Commission believes that Nasdaq has provided clear and comprehensible information on the overall operation of Dynamic M–ELO and the role of the machine-learning model and demonstrated that the proposal is consistent with the Act. Several related comments addressed this issue; these comments and Nasdaq’s responses are discussed below, followed by the Commission’s analysis. One commenter stated that the initial filing would establish ‘‘a dangerously vague standard for describing how exchange-hosted complex algorithmic 49 In addition to providing a statutory analysis in its filing, Nasdaq also acknowledges, above in Amendment No. 2, that the systems it will use to implement Dynamic M–ELO, including the Exchange’s model development and retraining processes, are SCI systems under Regulation SCI, see 17 CFR 242.1000 et seq., and thus, it will be responsible for compliance with Regulation SCI with respect to Dynamic M–ELO, including having appropriate policies and procedures. See supra note 33 and accompanying text. 50 15 U.S.C. 78f(b)(5). 51 15 U.S.C. 78f(b)(8). 52 See 17 CFR 201.700(b)(3). 53 15 U.S.C 78s(b)(1). E:\FR\FM\13SEN1.SGM 13SEN1 62858 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices ddrumheller on DSK120RN23PROD with NOTICES1 order types operate.’’ 54 In response to public comment, Nasdaq added more details describing the operation of Dynamic M–ELO to both the filing and public record since this proposed rule change was initially submitted to the Commission, and Nasdaq also provided additional legal analysis to support Dynamic M–ELO’s consistency with the Act. Prior to the filing of Amendment No. 1, a commenter stated that although ‘‘Nasdaq shared some of the 142 features of their formula,’’ Nasdaq should reveal all of these features so that prospective users may evaluate how the model works.55 Similarly, another commenter stated that the public cannot provide meaningful comment on the proposal without knowing all categories and parameters of the proposed Dynamic M– ELO.56 In its response to these comments, the Exchange, among other things, provided the specific 142 data elements that will be weighed by the machine-learning model as both an appendix to its first letter in response to comments,57 and as Exhibit 3B to its Amendment No. 1 filing. In response to the Exchange’s disclosures in Exhibit 3B of Amendment No. 1, one of these commenters stated that the list of data elements was not ‘‘clear and comprehensible’’ as is required by the Form 19–4, but rather ‘‘vague, confusing, and perfunctory.’’ 58 This commenter also stated that the disclosed data elements included unexplained terms (e.g., ‘‘baseline simulated,’’ ‘‘action simulated,’’ and ‘‘synthetic mark-out’’).59 In a subsequent comment letter, this commenter reiterated these points; the commenter specified that the commenter’s concern is that Nasdaq’s rule text does not disclose information about its methods for assessing market conditions and that ‘‘Nasdaq should carefully detail its methods in its rulebook, just like other exchanges have done, and Nasdaq should also thoroughly disclose its methods in its filing text.’’ 60 In its second response to 54 See Leuchtkafer Letter 1, supra note 48, at 1– 2. See also Letter from Joseph Saluzzi, Partner, Themis LLC, dated January 25, 2023, at 3 (‘‘Themis Letter’’) (questioning whether the complexity of Dynamic M–ELO is necessary). 55 See Themis Letter, supra note 54, at 2. 56 See Leuchtkafer Letter 1, supra note 48, at 1. 57 See Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated March 9, 2023, at Appendix A (‘‘Nasdaq First Response to Comments’’). 58 See Letter from R.T. Leuchtkafer, dated May 2, 2023, at 8–9 (‘‘Leuchtkafer Letter 2’’). 59 See id. 60 See Letter from R.T. Leuchtkafer dated May 30, 2023, at 3–5; 8–9 (‘‘Leuchtkafer Letter 3’’). This commenter also cites to the rules governing the Crumbling Quote Indicator and D-Limit order type on the Investors Exchange (‘‘IEX’’), as well as VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 comments 61 and the revised Exhibit 3b to Amendment No. 2, Nasdaq expanded and ‘‘simplified’’ the explanation of these 142 data elements. The Exchange also added to the proposed rule text a definition of the term ‘‘proprietary assessment of market conditions’’ to explain how the machine-learning model will evaluate those 142 data elements.62 Furthermore, Nasdaq attached, as Exhibit 3A to its proposed rule change, the White Paper written by its AI Core Development Team that explains, among other things, how Dynamic M– ELO’s machine-learning functions were developed and tested. The White Paper includes a general discussion of the type of model implemented in the proposed system, in this case a reinforcement learning model,63 as well as citations to academic research behind the Double Deep Q-Network algorithm that is the basis for the algorithm used in Nasdaq’s model.64 The White Paper also describes the ways in which Nasdaq’s implementation of the proposed model differs from the model and training in the academic research, providing both an English summary and a pseudocode description of differences in model training implemented by Nasdaq.65 One of the commenters stated that the White Paper is not easily understood by most market participants and that referencing the White Paper in the filing is an ‘‘unacceptable substitute’’ for a ‘‘plain English’’ explanation of the proposal in Form 19b–4.66 In response, Nasdaq explained that it drafted the filing to provide a general language from the Commission’s approval order for the D-Limit order type. See id. at 4. The commenter notes the level of detail with regard to how and when the D-Limit order type exercises its discretionary price-sliding that is set forth in the IEX Rulebook. See id. at 4; see also Themis Letter, supra note 54, at 2 (‘‘Another exchange, IEX, operates a smart logic called CQI (Crumbling Quote Indicator) which aims to protect orders from being adversely selected. IEX has published detailed notes on how the CQI is calculated.’’). Each proposal must be evaluated based on the specific facts and circumstances before the Commission. In this case, the Commission is only reviewing the proposed operation of Dynamic M–ELO and its machine-learning model. Accordingly, the level of detail provided in the IEX Rulebook for the D-Limit order type and Crumbling Quote Indicator—or the rulebooks for order types on other exchanges—does not determine whether Nasdaq has met its burden in this proposal. 61 See Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated May 18, 2023 at Appendix A (‘‘Nasdaq Second Response to Comments’’). 62 See supra note 25. 63 See White Paper Section 3.1. 64 See White Paper Section 4.1. 65 See White Paper, Section 7.2. 66 See Leuchtkafer Letter 2, supra note 58, at 3– 4; Leuchtkafer Letter 3, supra note 60, at 4–5; Letter from R.T. Leuchtkafer, dated August 11, 2023, at 8– 10 (‘‘Leuchtkafer Letter 4’’). PO 00000 Frm 00089 Fmt 4703 Sfmt 4703 understanding of Dynamic M–ELO and how it will behave, and the more detailed information and explanation in the White Paper are meant to support the filing.67 The Commission agrees with comments and the Exchange that there is an extent to which the proposed changes will introduce an unavoidable degree of uncertainty with respect to the use of these order types. The deep reinforcement learning model that will determine the dynamic holding periods for each symbol for M–ELO and M– ELO+CB orders will be implemented through established, non-discretionary methods,68 but it is so complex that its complete details are, for most intents and purposes, not readily intelligible, and it would be immensely difficult for the Exchange or any market participant to precisely predict the holding periods that will be generated by the model for any given symbol at any particular time. Nevertheless, as further discussed below, the Commission believes that the Exchange has provided information sufficient for the Commission and public to understand the design, operation, and limits of the proposed changes to these order types, and the role of the machine-learning model therein. While the holding periods under the proposal would be dynamic, Nasdaq has precisely articulated both the nature of changes that would be permissible under the proposal, and the limits to those changes. Nasdaq described when changes might occur (every thirty seconds throughout the trading day), the initial default holding period for all symbols (1.25 milliseconds), the permissible increments by which a holding period might change in each symbol (0.25 or 0.50 milliseconds), and the outer bounds of permissible holding period lengths (0.25 milliseconds at the short end, and 2.50 milliseconds at the long end). Nasdaq also described the conditions of ‘‘extraordinary instability’’ in a symbol when these holding periods would not apply, and when the holding periods would be overridden by the proposed ‘‘stability protection mechanism’’ (with a holding period of 12 milliseconds for at least 750 milliseconds). The Commission believes that these details provide sufficient information to understand the range of potential holding periods that may be applied when M–ELO or M–ELO+CB orders are entered or resting on the order book, the changes that may occur, and the limits to those changes. 67 See Nasdaq Second Response to Comments, supra note 61, at 3–5. 68 See infra notes 91–92 and accompanying text. E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices Nasdaq has also described the role of the machine-learning model in its proposal. The model will determine whether, by which increment, and in which direction to adjust the holding period for each symbol throughout the trading day.69 In its Form 19b–4, White Paper, and response letters, Nasdaq described the goals towards which the model is optimized: reducing mark-outs and increasing fill rates.70 Nasdaq’s White Paper includes a detailed discussion of model choice, development, and training, including citations to relevant other research.71 Nasdaq also provided several iterations of a list of data elements that the model will ingest and use, including a glossary defining terms used in the descriptions.72 Nasdaq affirmed that in operation during market hours, the data used would be calculated based on intraday market data.73 One version of these lists included Nasdaq’s estimates of the tendencies of data elements to affect model outcomes.74 Nasdaq’s White Paper also included an ‘‘explainability study’’ that assessed both the effects of individual data elements on model performance and the effects of interactions between individual data elements.75 Across its filing and incorporated exhibits, aspects of the model’s operations and design are described in different formats and with different levels of specificity—for example, the filing and exhibits include ‘‘plain English’’ descriptions, mathematical definitions, and pseudocode. Together, this set of information allows the Commission to understand the type of decision the model will implement, the goals the model aims to achieve, which model type is implemented and how it was developed, the range of data types and data sources used by the model, and 69 See supra notes 16–24 and accompanying text. e.g., Form 19b–4 at 9, White Paper Section 5, and Nasdaq First Response to Comments at 2. In its White Paper, Nasdaq provides mathematical definitions of fill rate for a period of time and markout by trade (White Paper at 5, Equations 1 and 2), as well as of the assessment made by the agent in the model’s reinforcement learning process (White Paper at 11, Equation 3). 71 See White Paper Sections 3–5. 72 See Exhibit 3B. As described below, these data elements are also those used in training and retraining the model. 73 See Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated September 6, 2023 (‘‘Nasdaq Third Response to Comments’’). 74 Nasdaq affirmed that, while this information was not included in all versions of the list of data elements, it remains accurate and valid. See id. at 3. 75 See White Paper Section 5.3. ddrumheller on DSK120RN23PROD with NOTICES1 70 See, VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 estimates of the manner in which those data may affect model outcomes. Nasdaq also explains how and when the machine-learning model will be retrained. Nasdaq will retrain the model weekly, outside of market hours. Retraining will incorporate market data obtained during the week from the equity consolidated data feeds and M– ELO order book. A retrained model will only be promoted to production if it improves upon the model objectives compared to the prevailing model.76 Furthermore, the Exchange explained that the machine-learning model is consistent in its behavior from day-today during periods when it is not undergoing retraining, ‘‘such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.’’ 77 The Exchange also stated in its initial response to comments that ‘‘[e]ven after the system undergoes retraining, which will occur on a weekly basis (and not during market hours), system behavior should not change dramatically or in unexpected ways from week-toweek.’’ 78 As noted above as well, the Exchange also represents that outside of set retraining periods, ‘‘the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.’’ 79 Nasdaq also will publish equity trader alerts when it anticipates that a model update may change mark-outs or fill rate by 10% or more in either direction.80 By including this set of information, Nasdaq has provided the Commission and public with information that allows them to understand how frequently the model will be retrained, the data used for retraining, and the criteria that will be used to determine whether to update the 76 For example, Nasdaq affirmatively states that if ‘‘a retrained model would be worse than the existing model in achieving its objectives, then the System will continue to use the existing model and discard the retrained model.’’ See Section III.A.1., supra. 77 See Nasdaq First Response to Comments, supra note 57, at 2–3. See also supra note 30. 78 See Nasdaq First Response to Comments, supra note 57, at 2–3. A commenter also noted that it was initially unclear when and how frequently the machine-learning model would retrain, stating that the White Paper set forth an analysis based on daily retraining, but the rule filing proposes weekly retraining. See Leuchtkafer Letter 2, supra note 58, at 4; Leuchtkafer Letter 3, supra note 60, at 2; Leuchtkafer Letter 4, supra note 66, at 9. In Amendment No. 2, the Exchange affirmatively represents that ‘‘the performance statistics for Dynamic M–ELO cited herein and in the White Paper are based upon data derived from weekly, not daily retrainings.’’ See supra note 38. 79 See supra note 46 and accompanying text. 80 See, e.g., Amendment No. 2 at 19–20. PO 00000 Frm 00090 Fmt 4703 Sfmt 4703 62859 production model based on retraining. This information allows the Commission to understand when the proposed model may change and when it will remain constant, the circumstances under which a change would be implemented, and circumstances under which the public will receive notice of significant changes in the model’s anticipated outcomes. In addition, a commenter stated their belief that the Exchange’s proposal for Dynamic M–ELO would result in the exercise of discretion by a national securities exchange because the machine-learning model’s decisions would vary over time based on the following: (1) varying parameter values; and (2) results of retraining cycles.81 The commenter stated that by making it possible for Dynamic M–ELO to behave differently when confronted by the same market conditions before and after the model is retrained, Nasdaq’s model would be exercising discretion that is more akin to a broker than an exchange.82 The commenter claimed that Dynamic M–ELO would operate outside of established non-discretionary methods, which require ‘‘fully disclosed procedures operating in a strictly linear, invariant, and deterministic fashion.’’ 83 Additionally, the commenter stated that Nasdaq would be exercising discretion with Dynamic M–ELO to alter a participant’s material order terms.84 The commenter claimed that Nasdaq would be using undisclosed data such as the 81 See Leuchtkafer Letter 1, supra note 48, at 2; Leuchtkafer Letter 2, supra note 58, at 4–6; Leuchtkafer Letter 3, supra note 60, at 5–7; Leuchtkafer Letter 4, supra note 66, at 4–8. 82 See Leuchtkafer Letter 1, supra note 48, at 2. See also Leuchtkafer Letter 3, supra note 60, at 5– 7. For example, the commenter believes that Dynamic M–ELO will ‘‘exercise individualized judgment’’ such that it can set a different time-inforce for the very same order presented in the very same market conditions on, for example, August 21 than it set on May 15, depending on the system’s undisclosed individualized judgments of market conditions and participant behavior from even days or weeks in the past. See Leuchtkafer Letter 4, supra note 66, at 6. 83 See Leuchtkafer Letter 2, supra note 58, at 5– 6. See also Leuchtkafer Letter 3, supra note 60, at 5–7. See also Leuchtkafer Letter 4, supra note 66, at 6–9 (’’ . . . . Nasdaq’s rulebook won’t set out the ‘totality of the discretionary feature’ (I believe it can’t, because the totality changes week-to-week and even minute-to-minute) and it won’t define the ‘hardcoded conditionality’ of its feature (again, I believe it can’t), and a market participant won’t be able to ‘recreate on its own’ what Dynamic M–ELO has done (participants can’t—it’s not even clear anyone will be able to, as discussed below) . . . . Dynamic M–ELO departs from decades of this progress. Its behavior will not be deterministic or invariant over time, and purposefully so.’’). 84 See Leuchtkafer Letter 2, supra note 58, at 8. See also Leuchtkafer Letter 3, supra note 60, at 5– 7. E:\FR\FM\13SEN1.SGM 13SEN1 62860 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices buyer and seller counts and recent trade sizes.85 Further, the commenter stated: that (a) by determining the universe of data the system consumes, (b) by programming how the system thinks, (c) by controlling and supplying the information with which it thinks, and (d) by setting the goals and programming the nature and extent of its actions, and when it does all this to determine (e) when and in which prescribed intervals to set an ever variable time-in-force term for an order, a term which (f) dictates when to expose an order to the market to find contra-side interest, then without question Nasdaq is exercising control, judgment, and discretion over its customer orders. In Amendment No. 1, Nasdaq added language to address these concerns.86 Among other things, Nasdaq stated that to the extent that the design of Dynamic M–ELO permits variation in the Holding Periods for such orders, it does so by design, and the ‘‘mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.’’ 87 Additionally, Nasdaq stated the following in its second response to comments: It is also worth noting that presently, exchanges like Nasdaq already employ nonlinear, non-deterministic functionalities, like the randomized timers it uses to resolve certain unavoidable race conditions that arise in the order handling process. Nasdaq employs these functionalities with the knowledge of the SEC, and without any suggestion that they somehow transform Nasdaq into a broker.88 ddrumheller on DSK120RN23PROD with NOTICES1 Furthermore, as noted above, the Exchange represents that outside of set retraining periods, ‘‘the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.’’ 89 The Exchange explains that outside of the set retraining periods ‘‘the System will behave predictably from day to day, such that its decisions when presented with given set of facts and circumstances in a given security on day 85 See Leuchtkafer Letter 2, supra note 58, at 8. See also Leuchtkafer Letter 3, supra note 60, at 5– 7; Leuchtkafer Letter 4, supra note 66, at 5–7. 86 See supra notes 42–46. 87 See supra note 45 and accompanying text. 88 See Nasdaq Second Response to Comments, supra note 61, at 5–9. See also Nasdaq First Response to Comments, supra note 57, at 5–7. 89 See supra note 46 and accompanying text. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 1 should be the same as they would be on day 2.’’ 90 Based on Nasdaq’s representations described above, Dynamic M–ELO would operate pursuant to predetermined, programmed procedures that would dictate order interaction and the terms for trading for each Dynamic M–ELO order entered on the Nasdaq trading facility. While the Exchange’s procedures include conditions that, if satisfied under certain circumstances, might result in different outcomes for different M–ELO orders, such conditions and circumstances, if predetermined, pre-defined, and programmed into the Exchange’s trading facility, would be considered established and not discretionary. For example, according to the Exchange, Dynamic M–ELO may apply different pre-determined weights over time to pre-determined factors it uses to determine whether and by how much to vary a Holding Period.91 In such an event, Dynamic M–ELO will operate pursuant to pre-determined procedures and programmed mathematical algorithm from which it cannot deviate to ‘‘achieve pre-defined and predisclosed objectives.’’ 92 Further, the procedures governing Dynamic M–ELO and use of M–ELO orders will be established before the beginning of each trading day. For example, Dynamic M– ELO will use preset methods to evaluate and weigh specific data elements to determine the dynamic holding periods. Such pre-set methods will be established during the prior retraining period, and outside regular trading hours, and will not vary intra-day until adjusted at the next retraining period. Given the pre-determined, programmed procedures and rules that Nasdaq has proposed to dictate trading for Dynamic M–ELO, the Commission does not believe that Dynamic M–ELO is designed to provide Nasdaq with judgement and flexibility, and therefore, discretion over the handling or execution of a M–ELO order entered on the Exchange. Unfair Discrimination Section 6(b)(5) of the Act requires that the rules of a national securities exchange not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers. In several letters, one commenter stated that Nasdaq inadequately explains how it will monitor and, if necessary, adjust Dynamic M–ELO to ensure no unfair 90 See supra note 29. supra note 45 and accompanying text. 92 See supra note 46 and accompanying text. 91 See PO 00000 Frm 00091 Fmt 4703 Sfmt 4703 discrimination.93 Initially, this commenter emphasized what they perceived to be silence on the part of Nasdaq with regard to whether Dynamic M–ELO will discriminate among categories of participant types.94 In its response to these comments, Nasdaq initially added a new representation to the filing in Amendment No. 1, stating that that Dynamic M–ELO is not designed to further the performance of any participant or any category of participant, but instead has twin objectives—the absolute values of markouts and fill rates. In Amendment No. 2, Nasdaq expanded on this representation by adding the following: Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M–ELO, as compared to the ‘‘static’’ M–ELO, for those firms that accounted for more than 95% of M–ELO activity on the Exchange during Q1 2022 . . . . The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M–ELO versus static M–ELO, with the volumeweighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant (i.e., nature of firm; size of firm) benefitted from Dynamic M–ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination. In Amendment No. 2, Nasdaq also affirmed that it will periodically review its model to ensure that it continues to perform in accordance with the Exchange’s rules and that it has not introduced any harmful bias in favor of or against any participant or class of participants.95 In response to the above, the commenter submitted a fourth comment letter, in which they questioned the approach Nasdaq took to demonstrate that there is not bias against any one 93 See Leuchtkafer Letter 3, supra note 60, at 7– 8; Leuchtkafer Letter 2, supra note 58, at 5; Leuchtkafer Letter 1, supra note 48, at 3. Nasdaq’s White Paper includes a ‘‘firm-level analysis’’ that ‘‘tried to identify patterns and trends that could potentially signify a systematic bias towards specific firms.’’ White Paper at 24. This analysis concluded that ‘‘Dynamic M–ELO will not result in systematic-biased execution towards any one firm.’’ Id. at 26. 94 See Leuchtkafer Letter 3, supra note 60, at 7– 8. 95 See Amendment No. 2 at fn. 34. E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices participant or class of participants.96 The commenter, among other things, expressed concern about Nasdaq conducting its analysis using data for firms that accounted for 95% of M–ELO activity during Q1 of 2022 rather than all M–ELO activity. The commenter states that Nasdaq did not describe how it determined the 5% of activity during that period to exclude from its analysis.97 For example, the commenter states that it is not clear whether Nasdaq excluded firms with large orders and trades, and the commenter opines that discarding any data could exclude activity that has qualitative or quantitative differences from the rest.98 In response to this comment, Nasdaq represented that it conducted a supplemental analysis of the initiallyexcluded data—which were the activity of the least-active M–ELO firms from the control period of its initial analysis—to confirm whether its initial conclusions held for those participants.99 Nasdaq explains that the individual variations among the previously excluded participants was higher than that for the original batch of data, but that, based on simulated data, each of these participants would have experienced the same or better fill rates during the testing period if they had utilized Dynamic M–ELO.100 Based on this supplemental data analysis, Nasdaq concluded that there is no apparent biases for the Dynamic M–ELO, even among the least active M–ELO participants.101 The Commission concludes that Nasdaq has adequately demonstrated that the proposal is not designed to permit unfair discrimination consistent with Section 6(b)(5) of the Act. Through the White Paper, amendments, and response letters, Nasdaq has demonstrated that it has analyzed the anticipated or simulated effects of the proposed change on all current M–ELO users, and that this work did not indicate that particular firms or classes of firms are anticipated to unfairly 96 See Leuchtkafer Letter 4, supra note 66, at 1– ddrumheller on DSK120RN23PROD with NOTICES1 4. 97 See id. at 2–3 (‘‘It seems Nasdaq trimmed its data before analyzing it for bias and constrained its analysis to ‘those firms that accounted for more than 95% of M–ELO activity on the Exchange during Q1 2022.’ (I assume Nasdaq used the same data defined in the Filing as the ‘Training Period’ for its analysis. Nasdaq doesn’t say so, however.) Nasdaq doesn’t describe the kind of M–ELO ‘activity’ it filtered the data for, and specifically whether it filtered on order or trade counts or order or trade volume or some combination of two or more of these categories, or on some other factor, before removing firms from its analysis.’’) 98 See id. at 2–3. 99 See id. at 5–6. 100 See id. 101 See id. VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 benefit from or be harmed by the proposed Dynamic M–ELO functionality. Prevention of Fraudulent and Manipulative Acts and Practices, Just and Equitable Principles of Trade, and the Protection of Investors and the Public Interest Section 6(b)(5) of the Act also requires that the rules of a national securities exchange be designed to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest. In Amendment No. 2, Nasdaq addressed whether the Dynamic M–ELO is designed to prevent fraudulent and manipulative acts and practices. Nasdaq states that the design of Dynamic M– ELO does not lend itself to potential manipulation by a single participant or a small group of participants because the machine-learning model makes determinations regarding Holding Periods based upon prevailing marketwide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M– ELOs involving that symbol. Nasdaq further states that manipulation of the machine-learning model would be difficult to accomplish given the large number of variables that factor into the machine-learning model’s decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, may vary over time. Furthermore, Nasdaq states that any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of effecting such manipulation. The Exchange, in Amendment No. 2, also sets forth representations regarding how it will surveil its market after Dynamic M–ELO is implemented. First, Nasdaq represents that it will review the machine-learning functionality and operation periodically to affirm that it continues to perform in accordance with the Exchange’s rules and has not introduced any harmful bias in favor of or against any participant or category of participants.102 Nasdaq also represents above that it will surveil for indications of manipulation and act accordingly if it detects such indications. The Commission finds that the proposed rule change is designed to prevent fraudulent and manipulative acts and practices consistent with 102 See PO 00000 supra note 41. Frm 00092 Fmt 4703 Sfmt 4703 62861 Section 6(b)(5) of the Act. The Commission agrees that aspects of the Dynamic M–ELO design reduce opportunities for manipulation or are likely to make manipulation costly or difficult. The model’s operation depends on 142 data elements, which are each likely to have effects on model outcomes of differing magnitudes and in different directions. Many of these data elements are also based on market-wide data, in some cases spanning periods of days,103 which are likely themselves difficult for market participants to manipulate. Given these design features, it appears likely that manipulating the duration of Dynamic M–ELO holding periods in any given symbol or group of symbols would be an extremely complex undertaking. In light of this complexity, and the size of M–ELO activity relative to the market for NMS stocks,104 Nasdaq’s assertion that the potential benefits of manipulating the dynamic holding periods for these order types would be outweighed by the cost and complexity of manipulation also appears reasonable. Nasdaq has also represented that it intends to surveil the proposed order types for manipulation. This ongoing surveillance, to ensure the appropriate use of Dynamic M–ELO by Exchange Members and behavior by the machine-learning model, is important to the successful implementation of Dynamic M–ELO and appears appropriately tailored to the accomplish the intent of the M–ELO and M– ELO+CB order types. Furthermore, the Commission finds that overall structure of Dynamic M– ELO—particularly, the static numerical constraints set forth in the proposed rule text—is designed in general, to protect investors and the public interest and promote just and equitable principles of trade pursuant to Section 6(b)(5) of the Act. As described above, the model will continuously engage in dynamic analysis of current market conditions during trading hours, and outside of market hours, it will retrain with the goal of improving the overall performance of Dynamic M–ELO. These dynamic aspects of the proposal, 103 See, e.g., Exhibit 3b. to the ‘‘M–ELO Monthly Report’’ published by Nasdaq for July 2023 (available at: https://www.nasdaq.com/docs/M-ELO-MonthlyReport (accessed September 2, 2023)), the average daily notional volume executed in M–ELO was $624,556,748. The average daily notional volume executed in July 2023 across the market for NMS stocks was about $523,769,246,196. See, e.g., Cboe, Historical Market Volume Data, available at: https://www.cboe.com/us/equities/market_ statistics/historical_market_volume/. The average daily notional volume in M–ELO for that month was approximately 0.12% (just over one-tenth of one percent) of the average daily notional volume across the entire NMS stock market. 104 According E:\FR\FM\13SEN1.SGM 13SEN1 62862 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices however, are constrained by the static numerical thresholds set forth in the proposed rule text. For example, the initial Holding Periods for each trading day will be 1.25 milliseconds, the overall range for any Holding Period must be between 0.25 and 2.50 milliseconds during normal market conditions, and the Holding Period can only change by either 0.25 or 0.50 milliseconds at each Change Event during normal market conditions. Regardless of how the model analyzes the current market or changes the weighting of the data elements as a result of its retraining, Dynamic M–ELO cannot operate outside of the static numerical ranges and limitations or minimums set forth in the rule text. As such, the Commission finds that Nasdaq has designed Dynamic M–ELO to operate in a manner that in general protects investors and the public interest and promotes just and equitable principles of trade in accordance with Section 6(b)(5) of the Act. ddrumheller on DSK120RN23PROD with NOTICES1 Compliance With SRO Recordkeeping and Reporting Obligations One commenter queried whether Nasdaq could maintain an adequate audit trail given the potential for frequently shifting Holding Periods for Dynamic M–ELO.105 In response, Nasdaq states that it will retain copies of each iteration of its system as part of its books and records and will disclose publicly statistics relating to Dynamic M–ELO performance.106 Nasdaq additionally represented that it will publish weekly and monthly Dynamic M–ELO performance statistics, which would include the weekly numbers of shares and trades in M–ELOs by symbol, weekly aggregated M–ELO share and trade data, and monthly aggregated block data, on Nasdaqtrader.com.107 Nasdaq also indicated it would add statistics to its existing M–ELO Monthly Report, which discloses quote stability by time horizon, about how frequently, on average, its system changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M–ELO and M–ELO+CB trading volumes.108 Nasdaq also added a representation to the filing, addressing how it would comply with its recordkeeping obligations.109 Nasdaq states that it will retain copies of each historical iteration of its models as part of its books and 105 See Leuchtkafer Letter 1, supra note 48, at 1. Nasdaq First Response to Comments, supra note 57, at 3. 107 See Section III.A.1., supra. 108 See id. 109 See id. 106 See VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time.110 Nasdaq also states that it will publish an equity trader alert in advance of deploying a retrained version of Dynamic M–ELO when Nasdaq anticipates the retrained version will produce results that differ materially from the prior version.111 Based on these representations, the Commission finds that Nasdaq has met its burden to demonstrate that it will comply with all relevant exchange recordkeeping requirements and obligations when it implements Dynamic M–ELO. In addition, the Commission notes that Nasdaq must comply with its reporting obligations under Rule 613 of Regulation NMS 112 and the National Market System Plan Governing the Consolidated Audited Trail (‘‘CAT NMS Plan’’) 113 with respect to Dynamic M– ELO, which requires it to record and electronically report to the central repository the material terms of each order and each reportable event.114 Nasdaq’s Obligation To File Proposed Rule Changes Relating to Dynamic M– ELO Prior to the filing of Amendment No. 1, a commenter stated that it was unclear what types of changes to the model would lead Nasdaq to seek approval from the Commission via an SRO rule filing.115 As explained above,116 Nasdaq represents that it will not modify the underlying structure of Dynamic M–ELO without first obtaining the Commission’s approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model may adjust the Holding Periods, the range of Holding Period durations available to M–ELOs and M–ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability. In contrast, the Exchange 110 See id. id. 112 See 17 CFR 242.613. 113 The CAT NMS Plan was approved by the Commission, as modified, on November 15, 2016. See Securities Exchange Act Release No. 79318 (November 15, 2016), 81 FR 84696 (November 23, 2016). 114 See 17 CFR 242.613(c)(7). 115 See Leuchtkafer Letter 1, supra note 48, at 2. 116 See Section II.A.1., supra. 111 See PO 00000 Frm 00093 Fmt 4703 Sfmt 4703 states that it will not seek Commission approval prior to retraining the model to adjust the weighting it applies to those data elements pursuant to the weekly retraining process. Section 19(b)(1) of the Act 117 and Rule 19b–4 thereunder 118 require an SRO to file a proposed rule change with the Commission whenever it seeks any proposed change in, addition to, or deletion from the rules governing the SRO and its members’ activities on the SRO. As discussed above, the proposal sets forth the specific data elements that Dynamic M–ELO will use during the trading day. Furthermore, the proposed rule change sets forth when the machine-learning model will retrain and the extent to which the retraining can and cannot cause the machine-learning model to update Dynamic M–ELO’s operation during subsequent trading days.119 In addition, the proposal sets forth the operation of Dynamic M–ELO, such as the potential range for a Holding Period, how often Dynamic M–ELO reevaluates market conditions for a given security to adjust a Holding Period, and the increment by which a Holding Period may be changed. Nasdaq represents that it will not change any of these aspects of the proposal or any other function of Dynamic M–ELO without first filing a proposed rule change.120 Nasdaq does, however, state that it would not file a proposed rule change in connection with the operation of the machine-learning model’s weekly retraining and the results of that process. Based on the foregoing, the Commission believes that Nasdaq has adequately responded to the commenter’s concern. Nasdaq will need to file a proposed rule to make any changes, additions, or deletions to the operation of Dynamic M–ELO as approved herein. Nasdaq has delineated when it would file a proposed rule change to alter the operation of Dynamic M–ELO, and when the machine-learning model would retrain and adjust the weighting it applies to the data elements without it filing a proposed rule change. Specifically, Nasdaq’s proposed rule change and rule text reflect the 142 data elements Dynamic M–ELO will consider when determining the Holding Period for a security and the goals Nasdaq will consider when weighing those data elements (i.e., reducing mark-outs and increasing fill rates) but does not set forth the relative weighting of each those individual data elements. Though 117 15 U.S.C. 78s(b)(1). CFR 240.19b–4. 119 See Section III.A.1., supra. 120 See supra note 30 and accompanying text. 118 17 E:\FR\FM\13SEN1.SGM 13SEN1 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices the structure of the proposal does not disclose of the exact weighting for each of the 142 data elements, it does set forth the two goals Nasdaq will consider when weighing those data elements initially and during each weekly retraining, which provides information as to how those 142 factors will be used in determining the Holding Period for a security. Based on how the proposed rule sets forth the goals that will govern each retraining, the Commission believes that Nasdaq’s delineation of when it would and would not file a proposed rule change to alter the operation of Dynamic M–ELO is consistent with Nasdaq’s rule filing obligation. The Commission agrees that the weekly retraining to optimize the weighting of the 142 data elements considered by Dynamic M–ELO to best achieve those goals within the rule’s parameters would not necessitate the filing of a proposed rule change with the Commission because those adjustments would be reasonably and fairly implied by the proposed rule. However, to the extent Nasdaq seeks to change, add to, or delete from the rule’s construct in connection with the weekly retraining, it would first be required to file a proposed rule change with the Commission. V. Solicitation of Comments on Amendment No. 2 to the Proposed Rule Change Interested persons are invited to submit written data, views, and arguments concerning whether Amendment No. 2 is consistent with the Act. Comments may be submitted by any of the following methods: ddrumheller on DSK120RN23PROD with NOTICES1 Electronic Comments • Use the Commission’s internet comment form (https://www.sec.gov/ rules/sro.shtml); or • Send an email to rule-comments@ sec.gov. Please include file number SR– NASDAQ–2022–079 on the subject line. Paper Comments • Send paper comments in triplicate to Secretary, Securities and Exchange Commission, 100 F Street NE, Washington, DC 20549–1090. All submissions should refer to file number SR–NASDAQ–2022–079. This file number should be included on the subject line if email is used. To help the Commission process and review your comments more efficiently, please use only one method. The Commission will post all comments on the Commission’s internet website (https://www.sec.gov/ rules/sro.shtml). Copies of the submission, all subsequent amendments, all written statements VerDate Sep<11>2014 17:37 Sep 12, 2023 Jkt 259001 with respect to the proposed rule change that are filed with the Commission, and all written communications relating to the proposed rule change between the Commission and any person, other than those that may be withheld from the public in accordance with the provisions of 5 U.S.C. 552, will be available for website viewing and printing in the Commission’s Public Reference Room, 100 F Street NE, Washington, DC 20549 on official business days between the hours of 10:00 a.m. and 3:00 p.m. Copies of the filing also will be available for inspection and copying at the principal office of the Exchange. Do not include personal identifiable information in submissions; you should submit only information that you wish to make available publicly. We may redact in part or withhold entirely from publication submitted material that is obscene or subject to copyright protection. All submissions should refer to File Number SR–NASDAQ–2022– 079, and should be submitted on or before October 4, 2023. VI. Accelerated Approval of Proposed Rule Change, as Modified by Amendment No. 2 The Commission finds good cause to approve the proposed rule change prior to the 30th day after the date of publication of Amendment No. 2 in the Federal Register. Amendment No. 2 does not include any material changes to the operation of the proposed Dynamic M–ELO and its machinelearning model. In Amendment No. 2, the Exchange: (1) adds the defined term ‘‘proprietary assessment of market conditions’’ to the proposed rule text, which consolidates certain details and explanations about how the machinelearning model would operate from prior versions into a single defined term; (2) revises the list of factors provided in Exhibit 3b to include expanded and ‘‘simplified’’ explanations of the terminology used therein; (3) adds a representation that the systems used to operate Dynamic M–ELO and machine-learning model are ‘‘SCI Systems’’ and thus subject to compliance with Regulation SCI; and (4) expands the legal analysis to address comments regarding unfair discrimination and the exercise of impermissible discretion by the Exchange. The Commission finds that Amendment No. 2 raises no novel regulatory issues that have not previously been subject to comment and is reasonably designed to prevent fraudulent and manipulative acts and PO 00000 Frm 00094 Fmt 4703 Sfmt 4703 62863 practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be unfairly discriminatory, or impose an unnecessary or inappropriate burden on competition. Amendment No. 2 does not alter the proposed operation or any material features of Dynamic M–ELO, which operation and features have been subject to two rounds of public comment. In response to public comment, the revisions to the proposal contained within Amendment No. 2 provide additional clarification and details regarding how Dynamic M– ELO and the machine-learning model will operate, as well as additional legal analysis to support the Exchange’s position that the proposal is consistent with the Act. Accordingly, pursuant to Section 19(b)(2) of the Act,121 the Commission finds good cause to approve the proposed rule change on an accelerated basis prior to the 30th day after publication of notice of the filing of Amendment No. 2 in the Federal Register. VII. Conclusion It is therefore ordered, pursuant to Section 19(b)(2) of the Act,122 that the proposed rule change (SR–NASDAQ– 2022–079), as modified by Amendment No. 2, be, and it hereby is, approved on an accelerated basis. For the Commission, by the Division of Trading and Markets, pursuant to delegated authority.123 Sherry R. Haywood, Assistant Secretary. [FR Doc. 2023–19728 Filed 9–12–23; 8:45 am] BILLING CODE 8011–01–P SECURITIES AND EXCHANGE COMMISSION [Release No. 34–98320; File No. SR–PHLX– 2023–41] Self-Regulatory Organizations; Nasdaq PHLX LLC; Notice of Filing and Immediate Effectiveness of Proposed Rule Change To Amend Options 3, Section 13 Concerning PIXL September 7, 2023. Pursuant to section 19(b)(1) of the Securities Exchange Act of 1934 (‘‘Act’’),1 and Rule 19b–4 thereunder,2 notice is hereby given that on August 30, 2023, Nasdaq PHLX LLC (‘‘Phlx’’ or ‘‘Exchange’’) filed with the Securities and Exchange Commission 121 15 U.S.C. 78s(b)(2). U.S.C. 78s(b)(2). 123 17 CFR 200.30–3(a)(12). 1 15 U.S.C. 78s(b)(1). 2 17 CFR 240.19b–4. 122 15 E:\FR\FM\13SEN1.SGM 13SEN1

Agencies

SECURITIES AND EXCHANGE COMMISSION

[Federal Register Volume 88, Number 176 (Wednesday, September 13, 2023)]
[Notices]
[Pages 62850-62863]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-19728]


-----------------------------------------------------------------------

SECURITIES AND EXCHANGE COMMISSION

[Release No. 34-98321; File No. SR-NASDAQ-2022-079]


Self-Regulatory Organizations; The Nasdaq Stock Market LLC; 
Notice of Filing of Amendment No. 2 and Order Granting Accelerated 
Approval of a Proposed Rule Change, as Modified by Amendment No. 2, To 
Amend Rules 4702(b)(14) and (b)(15) Concerning Dynamic M-ELO Holding 
Period

September 7, 2023.

I. Introduction

    On December 21, 2022, The Nasdaq Stock Market LLC (``Nasdaq'' or 
``Exchange'') filed with the Securities and Exchange Commission 
(``Commission''), pursuant to Section 19(b)(1) of the Securities 
Exchange Act of 1934 (``Act'') \1\ and Rule 19b-4 thereunder,\2\ a 
proposed rule change to replace the static holding period requirements 
for Midpoint Extended Life Orders and Midpoint Extended Life Orders 
Plus Continuous Book with dynamic holding periods. The proposed rule 
change was published for comment in the Federal Register on January 10, 
2023.\3\ On February 22, 2023, pursuant to Section 19(b)(2) of the 
Act,\4\ the Commission designated a longer period within which to 
approve the proposed rule change, disapprove the proposed rule change, 
or institute proceedings to determine whether to disapprove the 
proposed rule change.\5\ On March 9, 2023, the Exchange filed Amendment 
No. 1 to the proposed rule change, which amended and superseded the 
proposed rule change as originally filed. On April 7, 2023, the 
Commission provided notice of filing of Amendment No. 1 and instituted 
proceedings to determine whether to approve or disapprove the proposed 
rule change, as modified by Amendment No. 1.\6\ On July 6, 2023, 
pursuant to Section 19(b)(2) of the Act,\7\ the Commission designated a 
longer period on proceedings to determine whether to approve or 
disapprove the proposed rule change.\8\ On July 18, 2023, the Exchange 
filed Amendment No. 2 to the proposed rule change, which amended and 
superseded the proposed rule change as amended by Amendment No. 1. The 
Commission received comments on the proposed rule change.\9\ The 
Commission is publishing this Notice and Order to solicit comment on 
Amendment No. 2 in Sections II and III below, which sections are being 
published verbatim as filed by the Exchange, and to approve the 
proposed rule change, as modified by Amendment No. 2, on an accelerated 
basis.
---------------------------------------------------------------------------

    \1\ 15 U.S.C. 78s(b)(1).
    \2\ 17 CFR 240.19b-4.
    \3\ See Securities Exchange Act Release No. 92844 (January 4, 
2023), 88 FR 1438.
    \4\ 15 U.S.C. 78s(b)(2).
    \5\ See Securities Exchange Act Release No. 96963, 88 FR 12710 
(February 28, 2023).
    \6\ See Securities Exchange Act Release No. 97263, 88 FR 22498 
(April 13, 2023).
    \7\ 15 U.S.C. 78s(b)(2).
    \8\ See Securities Exchange Act Release No. 97844, 88 FR 44423 
(July 12, 2023).
    \9\ All comments received by the Commission on the proposed rule 
change are available on the Commission's website at: https://www.sec.gov/comments/sr-nasdaq-2022-079/srnasdaq2022079.htm.
---------------------------------------------------------------------------

II. Self-Regulatory Organization's Statement of the Terms of Substance 
of the Proposed Rule Change

    The Exchange proposes to amend Rules 4702(b)(14) and (b)(15) of the 
Exchange's Rulebook to replace the static holding period requirements 
for Midpoint Extended Life Orders and Midpoint Extended Life Orders 
Plus Continuous Book with dynamic holding periods. This Amendment No. 2

[[Page 62851]]

supersedes the original filing and Amendment No. 1 \10\ in their 
entireties.
---------------------------------------------------------------------------

    \10\ See SR-Nasdaq-2022-079 Amendment No. 1 (March 9, 2023), at 
https://www.sec.gov/comments/sr-nasdaq2022-079/srnasdaq2022079-20159016-327215.pdf.
---------------------------------------------------------------------------

    The text of the proposed rule change is available on the Exchange's 
website at https://listingcenter.nasdaq.com/rulebook/nasdaq/rules, at 
the principal office of the Exchange, and at the Commission's Public 
Reference Room.

III. Self-Regulatory Organization's Statement of the Purpose of, and 
Statutory Basis for, the Proposed Rule Change

    In its filing with the Commission, the Exchange included statements 
concerning the purpose of and basis for the proposed rule change and 
discussed any comments it received on the proposed rule change. The 
text of these statements may be examined at the places specified in 
Item IV below. The Exchange has prepared summaries, set forth in 
sections A, B, and C below, of the most significant aspects of such 
statements.

A. Self-Regulatory Organization's Statement of the Purpose of, and 
Statutory Basis for, the Proposed Rule Change

1. Purpose
    The Exchange proposes to amend Rules 4702(b)(14) and (15) of the 
Exchange's Rulebook to replace the static 10 millisecond holding period 
requirements for its Midpoint Extended Life Order (``M-ELO'') and 
Midpoint Extended Life Order Plus Continuous Book (``M-ELO+CB'') Order 
Types with dynamic holding periods (``Dynamic M-ELO and M-ELO+CB'' or 
collectively, ``Dynamic M-ELO'').
Background
    In 2018, the Exchange introduced the M-ELO, which is a Non-
Displayed Order priced at the Midpoint between the National Best Bid 
and Offer (``NBBO'') and which is eligible for execution only against 
other eligible M-ELOs and only after a minimum of one-half second 
passes from the time that the System accepts the order (the ``Holding 
Period'').\11\ In 2019, the Exchange introduced the M-ELO+CB, which 
closely resembles the M-ELO, except that a M-ELO+CB may execute at the 
midpoint of the NBBO, not only against other eligible M-ELOs (and M-
ELO+CBs), but also against Non-Displayed Orders with Midpoint Pegging 
and Midpoint Peg Post-Only Orders (``Midpoint Orders'') that rest on 
the Continuous Book for at least one-half second and have Trade Now 
enabled.\12\
---------------------------------------------------------------------------

    \11\ See Securities Exchange Act Release No. 34-82825 (March 7, 
2018), 83 FR 10937 (March 13, 2018) (SR-NASDAQ-2017-074) (``M-ELO 
Approval Order'').
    \12\ See Securities Exchange Act Release No. 34-86938 (September 
11, 2019), 84 FR 48978 (September 17, 2019) (SR-NASDAQ-2019-048) 
(``M-ELO+CB Approval Order'').
---------------------------------------------------------------------------

    When the Exchange designed M-ELO, it originally set the length of 
the Holding Period at one-half second because it determined that this 
time period would be sufficient to ensure that likeminded investors 
would interact only with each other, and with minimal market impacts. 
The Exchange believed that the longer length of the M-ELO Holding 
Period and its simplicity in design would provide greater protection 
for participants than they could achieve through competing delay 
mechanisms.
    In 2020, however, the Exchange shortened the length of the Holding 
Period to 10 milliseconds.\13\ The Exchange did so after studying two 
years of actual use and performance of M-ELOs, as well as customer 
feedback. That is, the Exchange came to understand that, while users of 
M-ELO and M-ELO+CB are less concerned with achieving rapid executions 
of their Orders than are other participants, they are not indifferent 
about the length of time in which their M-ELOs and M-ELO+CBs must wait 
before they are eligible for execution. Indeed, participants informed 
the Exchange that in certain circumstances, such as when they sought to 
trade symbols that on average had a lower time-to-execution than a 
half-second, they were reticent to enter M-ELOs or M-ELO+CBs. They 
indicated that the associated Holding Periods for these Order Types 
were longer than necessary to achieve the desired protections and that, 
during the residual portion of the Holding Periods, they risked losing 
out on favorable execution opportunities that would otherwise be 
available to them had they placed a non-MELO order.
---------------------------------------------------------------------------

    \13\ See Securities Exchange Act Release No. 34-88743 (April 24, 
2020), 85 FR 24068 (April 30, 2020) (SR-NASDAQ-2020-011) (``M-ELO 
Timer Approval Order'').
---------------------------------------------------------------------------

    Based upon this feedback, the Exchange studied the potential 
effects of reducing the length of the Holding Periods for both M-ELOs 
and M-ELO+CBs (as well as for Midpoint Orders that would execute 
against M-ELO+CBs). Ultimately, the Exchange determined that it could 
reduce the Holding Periods to 10 milliseconds without compromising the 
protective power that M-ELO and M-ELO+CB are intended to provide to 
participants and investors.\14\ Thus, the Exchange determined that 
shortening the Holding Periods to 10 milliseconds for M-ELOs and M-
ELO+CBs would increase the efficacy of the mechanism while not 
undermining the power of those Order Types to fulfill their underlying 
purpose of minimizing market impacts. At the same time, the Exchange 
determined that a reduction in the Holding Periods to 10 milliseconds 
would dramatically add to the circumstances in which M-ELOs and M-
ELO+CBs would be useful to participants. In its M-ELO Timer Approval 
Order, the Commission agreed with the Exchange:
---------------------------------------------------------------------------

    \14\ The Exchange examined each of its historical M-ELO 
executions to determine at what Midpoints of the NBBO the M-ELOs 
would have executed if their Holding Periods had been shorter than 
one-half second (500 milliseconds). After examining the historical 
effects of shorter Holding Periods of between 10 milliseconds and 
400 milliseconds, the Exchange determined that a reduction of the M-
ELO Holding Period to as short as 10 milliseconds would have caused 
an average impact on mark-outs of only 0.10 basis points (across all 
symbols). In other words, compared to the execution price of an 
average M-ELO with a one-half second Holding Period, the Exchange 
found that a M-ELO with a 10 millisecond Holding Period would have 
had an average post-execution impact that was only a tenth of a 
basis point per share--a difference in protective effect that is 
immaterial. See Nasdaq, ``The Midpoint Extended Life Order (M-ELO); 
M-ELO Holding Period,'' available at https://www.nasdaq.com/articles/the-midpoint-extended-life-order-m-elo%3A-m-elo-holding-period-2020-02-13 (analyzing effects of shortened Holding Periods on 
M-ELO performance).
---------------------------------------------------------------------------

    The Commission notes that, with the proposed ten-millisecond 
Holding Period and Resting Period, M-ELOs and M-ELO+CBs would continue 
to be optional order types that are available to investors with longer 
investment time horizons, including institutional investors. The 
Commission also believes that the proposal could make M-ELOs and M-
ELO+CBs more attractive for securities that on average have a time-to-
execution of less than one-half second and, for investors who currently 
do not use M-ELOs and M-ELO+CBs for these securities, provide optional 
order types that could enhance their ability to participate effectively 
on the Exchange. The Commission notes that, if market participants 
determine that the proposal would make M-ELOs and M-ELO+CBs less 
attractive for their particular investment objectives, such market 
participants may elect to reduce or eliminate their use of these 
optional order types. Moreover, as noted above, the Exchange will 
continue to conduct real-time surveillance to monitor the use of M-ELOs 
and M-ELO+CBs to ensure that such usage remains appropriately tied to 
the intent of the order types. If, as a result of such surveillance, 
the Exchange determines that the shortened

[[Page 62852]]

Holding Period does not serve its intended purpose or adversely impacts 
market quality, the Exchange would seek to make further 
recalibrations.\15\
---------------------------------------------------------------------------

    \15\ M-ELO Timer Approval Order, supra, at 85 FR 24069.
---------------------------------------------------------------------------

    For similar reasons and with even better potential results for 
participants, the Exchange now proposes to further refine the length of 
the Holding Periods for M-ELOs and M-ELO+CBs, this time through the 
application of innovative and patent pending machine learning 
technology.
Dynamic M-ELO
    After receiving feedback from participants that even 10 millisecond 
Holding Periods for M-ELO and M-ELO+CB may, at times, exceed what is 
necessary to accomplish the underlying intent of these Order Types, the 
Exchange began to experiment with making further refinements to the 
duration of the Holding Periods. Ultimately, the Exchange concluded 
that shorter Holding Periods could achieve the same, if not better 
results for participants in terms of mark-outs, but not in all 
circumstances. That is, where prices of the underlying securities are 
stable, and not subject to imminent unfavorable changes, M-ELOs and M-
ELO+CBs face lower risks of confronting spread-crossing orders, such 
that shorter Holding Periods could suffice to protect M-ELOs and M-
ELO+CB from such orders. In periods of heightened price volatility, 
however, M-ELOs and M-ELO+CBs also face heightened risks, such that 
longer Holding Periods would continue to be beneficial in protecting M-
ELOs and M-ELO+CBs from such risks. Thus, the Exchange determined that 
another across-the-board reduction of the static 10 millisecond Holding 
Periods would be sub-optimal because it could impact the performance of 
the M-ELO and M-ELO+CB Order Types during periods of heightened 
volatility.
    In light of these observations, the Exchange tasked its artificial 
intelligence and machine learning laboratory (the ``AI Core Development 
Group'') to explore whether it could employ these innovative 
technologies to optimize the length of M-ELO and M-ELO+CB Holding 
Periods during various states of price volatility, and then to vary the 
lengths of the Holding Periods dynamically during the lifecycles of M-
ELOs and M-ELO+CBs, with the objectives of improving the performance of 
these Order Types while also further reducing opportunity costs.
    As the Exchange explains in greater depth in the attached White 
Paper,\16\ the AI Core Development Group proceeded to develop an 
artificial intelligence-based timer control system that will achieve 
these objectives.\17\ The AI Core Development Group did so by using 
reinforcement learning techniques--machine learning paradigms which 
develop optimal solutions to problems over time by taking actions to 
solve them, generating feedback on the results of such actions, 
applying that feedback to direct and improve the next round of 
solutions, and then repeating the feedback loop until the paradigm 
achieves optimized solutions.
---------------------------------------------------------------------------

    \16\ See Diana Kafkes et al., ``Applying Artificial Intelligence 
& Reinforcement Learning Methods Towards Improving Execution 
Outcomes,'' SSRN, October 19, 2022, available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4243985 (attached hereto 
[sic] without modification from the prior version as Exhibit 3(a)) 
(the ``White Paper'').
    \17\ Although the AI Core Development Group acknowledges that an 
optimal Holding Period would update with every incoming order, it 
determined that training a reinforcement learning model on every 
order would be too difficult to program and too difficult to 
implement given the nanosecond latency requirements of the Exchange. 
The Group then investigated more feasible update cadences and 
determined the point at which optimal outcomes were best balanced 
with the level of programming and implementation difficulty to be 
between 15 and 30 second updates. Ultimately, the Group chose a 30 
second update cadence to give the model the greatest opportunity to 
learn between potential actions.
---------------------------------------------------------------------------

    In this instance, the AI Core Development Group applied 
reinforcement learning techniques to a simulation of the M-ELO Book 
that it constructed using a representative data set from the first 
quarter of 2022 (the ``Training Period''). The Training Period data 
consisted of 380 out of the 6,257 symbols on the M-ELO Book (accounting 
for approximately 67 percent of M-ELO volume). The symbols chosen 
reflect both actively-traded and thinly-traded securities, and both 
low-priced and high-priced securities.
    The AI Core Development Group then developed a machine learning 
model and applied it to the Training Period data. The Group programmed 
the model to value the achievement of higher fill rates or lower mark-
outs than that which occurred in a historical simulation of M-ELOs and 
M-ELO+CBs involving the Training Period data.\18\ The Group then 
programmed the model to seek to achieve its goals by taking one of five 
possible actions with respect to the duration of the Holding Periods at 
30 second intervals \19\ for each symbol during each trading day of the 
Training Period. That is, at each 30 second internal, the model 
evaluated market conditions for each symbol over the prior 30 second 
period and either kept the Holding Periods the same, increased/
decreased them by 0.25 milliseconds, or increased/decreased them by 
0.50 milliseconds.\20\ After each decision-making round, the model 
utilized the results to inform its actions at the next 30 second 
increment.
---------------------------------------------------------------------------

    \18\ As the White Paper explains, the Group developed a model to 
simulate activity on the Exchange involving M-ELOs and M-ELO+CBs 
during the Training Period. See White Paper, supra, at 10.
    \19\ See id.
    \20\ The AI Core Development Group experimented with a range of 
permissible Holding Period durations. Ultimately, it concluded that 
it could produce better outcomes for M-ELO and M-ELO+CB participants 
than the existing approach using Holding Periods as low as 0.25 
milliseconds and as high as 2.5 milliseconds, under normal market 
conditions.
---------------------------------------------------------------------------

    In making its decisions, the model (again, drawing upon a 
combination of historical SIP and M-ELO-specific data) considered 142 
categories of data points.\21\ A confluence of data points that 
correlated with an increase in volatility tended to cause the model to 
increase the durations of Holding Periods, including increases in the 
standard deviation of NBBO prices, the number of unique participants 
placing sell orders on M-ELO and M-ELO+CB, and the volume-weighted 
average of the NBBO spread. Conversely, a confluence of data points 
that correlated with greater price stability tended to cause the model 
to decrease the durations of Holding periods, such as an increase in 
the median and max number of shares per trade and the number of resting 
bids left in the M-ELO and M-ELO+CB Book.
---------------------------------------------------------------------------

    \21\ Nasdaq attaches a full list of these data elements 
(attached hereto [sic] as ``Exhibit 3(b))'', along with an 
observation of the strength of the correlations that currently exist 
between changes to those data values and decisions the system makes 
to set the duration of Holding Periods at any given time. The 
Exchange notes that the version of this list attached to this 
Amendment No. 2 supersedes prior versions attached to prior versions 
of this filing. This version of the list includes expanded 
explanations of the terminology used therein. See also White Paper, 
supra, at 31, for a description of these features.
---------------------------------------------------------------------------

    The AI Core Development Team produced variations of its model that 
prioritized achievement of the lowest mark-outs, the highest fill 
rates, and a blend of these two objectives.\22\ Through

[[Page 62853]]

a process of learning and experimentation involving a combination of 
historical and simulated data, the AI Core Development Group settled on 
a Dynamic M-ELO model that achieved substantial simulated performance 
improvements for users of M-ELO and M-ELO+CB--both in terms of mark-
outs and fill rates--as compared to the static 10 millisecond Holding 
Periods. As the White Paper explains in greater detail, Dynamic M-ELO 
yielded an average combined volume-weighted (simulated) improvement of 
31.7 percent, including a 20.3 percent increase in fill rates and a 
11.4 percent reduction in mark-outs.\23\ The White Paper provides a 
more fulsome explanation of these improvements.\24\
---------------------------------------------------------------------------

    \22\ The AI Core Development Group also applied to the model a 
paradigm called ``retraining'' to combat the degradation of model 
performance that can otherwise occur as the reference data it uses 
for initial comparison becomes stale. Finally, the AI Core 
Development group added a stability protection mechanism to the 
model to provide maximum production to participants in the event 
that the model observes extraordinary levels of instability in the 
National Best Bid and Offer during the prior three seconds as 
compared to reference data. When the model detects such instability, 
it is programmed to increase the length of the Holding Period to 12 
milliseconds for a period of 750 milliseconds.
    \23\ See White Paper, supra, at 22.
    \24\ See id.
---------------------------------------------------------------------------

    Based upon these exciting results, the Exchange now proposes to 
amend Rule 4702(b)(14) and (15) to replace the static 10 millisecond 
timers applicable to M-ELO and M-ELO+CB with Dynamic M-ELO Holding 
Periods. Using the Exchange's ``proprietary assessment of market 
conditions'' \25\ and patent pending technology, the Dynamic M-ELO 
system will evaluate and, as it deems necessary, adjust the length of 
the Holding Periods for each symbol comprising M-ELOs and M-ELO+CBs 
(and Midpoint Orders on the Continuous Book that opt to interact with 
M-ELO+CBs after resting on the Book) every 30 seconds throughout the 
Market Hours (each such 30 second interval, a ``Change Event''). In so 
doing, Dynamic M-ELO will help participants to achieve a more optimized 
blend of the underlying purposes of the M-ELO and M-ELO+CB Order Types: 
protection against adverse selection (low mark-outs) without 
sacrificing opportunities to achieve high-quality executions (high fill 
rates).
---------------------------------------------------------------------------

    \25\ As set forth in the proposed rule text, the phrase 
``proprietary assessment of market conditions'' refers to the 
Exchange's evaluation of prevailing market conditions for a given 
symbol using an algorithm programmed to set a Holding Period 
duration which, at each Change Event, achieves an optimal blend of 
two objectives: maximization of M-ELO fill rates; and minimization 
of M-ELO mark-out rates. As the rule text states and as is discussed 
below, the algorithm ingests and analyzes 142 data points, which the 
Exchange identifies and describes in Exhibit 3b hereto. The Exchange 
derives these data from a combination of public data and M-ELO data 
feeds. Furthermore, the Exchange conducts weekly re-trainings of the 
algorithm, outside of Market Hours, to improve its performance 
relative to the immediately preceding period (in terms of the two 
aforementioned objectives). The Exchange deploys a retrained version 
of the algorithm only if it determines that doing so will, in fact, 
improve its performance relative to the immediately preceding 
period. The Exchange provides further information about the 
algorithm and the retraining process in a White Paper attached 
hereto [sic] as Exhibit 3a.
---------------------------------------------------------------------------

    A proposed M-ELO or M-ELO+CB with a Dynamic Holding Period will 
operate as follows. At the outset of Market Hours (approximately 
9:30:00 a.m.), the Exchange will impose initial Holding Periods of 1.25 
milliseconds for M-ELOs and M-ELO+CBs in all symbols. Thereafter, 
Holding Periods for a given symbol will become eligible to change 
dynamically from the initial duration beginning at 9:30:30 a.m. and 
then at 30 second intervals thereafter during Market Hours. The 
Exchange will then apply to the M-ELO or M-ELO+CB Order a Holding 
Period that is of the duration that prevailed at the time of entry. For 
example, if participant A enters a M-ELO for symbol XYZ at 9:30:25 
a.m., then Holding Period for that M-ELO will be 1.25 milliseconds. If 
at 9:30:30:00 a.m., the System decides to lower the duration of the 
Holding Period by 0.50 milliseconds, and then participant B enters a M-
ELO for symbol XYZ at 9:30:45 a.m., then the System will assign a 0.75 
millisecond Holding Period to participant B's M-ELO. To be clear, the 
System will determine Dynamic M-ELO Holding Periods independently for 
M-ELOs and M-ELO+CBs in each symbol.
    During normal market conditions, the range of potential Holding 
Period durations for M-ELOs and M-ELO+CBs will be between 0.25-2.50 
milliseconds, with the Holding Period duration being eligible to change 
by increments of either 0.25 or 0.50 milliseconds at each Change Event. 
Thus, if the Holding Period for a M-ELO in symbol XYZ is set at 0.75 
milliseconds at 2:22:11 p.m., and at 2:22:41 p.m., the System 
determines to increase the duration of the Holding Period, it may do so 
only by 0.25 or 0.50 milliseconds during that event.
    When a Change Event occurs, and the System determines to adjust the 
duration of a Holding Period for a symbol, that adjustment will apply, 
not only to all M-ELOs and M-ELO+CBs for that symbol entered within the 
30 second period after the Change Event occurs, but also to M-ELOs and 
M-ELO+CBs entered prior to the Change Event with unexpired Holding 
Periods (with applicability retroactive to the time of Order 
acceptance). Thus, if a participant enters a M-ELO in symbol XYZ at 
1:14:299 p.m., and the prevailing Holding Period applicable to that M-
ELO is 2 milliseconds, and at 1:14:30 p.m., the System modifies the 
Holding Period to be 1.5 milliseconds, then the M-ELO will become 
eligible to execute at 1:14:3005 p.m. This is the case because the M-
ELO will have already expended 1 millisecond of its Holding Period as 
of the time of the Change Event; thereafter, the M-ELO will need to 
rest only another 0.5 milliseconds to become eligible to execute under 
the new 1.5 millisecond Holding Period (as measured from 1:14:299 
p.m.). This last feature ensures that the M-ELO Book maintains time 
priority among M-ELOs and M-ELO+CBs in a dynamic environment. That is, 
it ensures that no M-ELO or M-ELO+CB with an unexpired Holding Period 
at the time of a Change Event will end up becoming eligible to execute 
later than a M-ELO entered after the Change Event which has a shorter 
Holding Period applicable to it.
    If at any time, the System detects extraordinary instability in a 
symbol, then the System will activate a ``stability protection 
mechanism'' to provide an extra layer of protection to M-ELO and M-ELO 
users from the heightened risks of adverse selection that exists during 
such periods of instability.\26\ The stability protection mechanism 
will override the prevailing Holding Periods for M-ELOs and M-ELO+CBs 
in a symbol experiencing extraordinary instability and immediately 
increase the duration of those Holding Periods to 12 milliseconds for a 
period of 750 milliseconds. The System may activate the stability 
protection mechanism even between Change Events. The System will 
evaluate, at each NBBO update, whether market conditions remain 
extraordinarily unstable and, if so, it will restart the 750 
millisecond Stability Protected Period and maintain the 12 millisecond 
Holding Period until conditions stabilize. Once the System determines 
that market conditions have stabilized (i.e., all measurements for the 
symbol are at or below the threshold value throughout the duration of 
the prevailing Stability Protected Period), the System will revert the 
duration of the Holding Periods to that which prevailed as of the 
Change Event that

[[Page 62854]]

occurred immediately prior to the activation of the stability 
protection mechanism or, if the stability protection mechanism was 
active when a Change Event occurred, to the duration selected at the 
immediately preceding Change Event. The System will then proceed to 
reevaluate the duration of the Holding Periods as per the regular 
schedule of Change Events.
---------------------------------------------------------------------------

    \26\ For purposes of this Rule, the System determines that 
``extraordinary instability'' for a symbol exists through 
observations it makes following every change in the NBBO for that 
symbol that occurs during the trading day. When the NBBO changes, 
the System looks back at the prior three seconds of trading and 
measures the difference between the highest and the lowest NBBO 
midpoint values that occurred during that period, and then it 
compares that measurement to a threshold value for the symbol. The 
System concludes that extraordinary instability exists for a symbol 
if the measurement exceeds the threshold value. The threshold value 
for a symbol, in turn, is the difference between the highest and the 
lowest NBBO midpoint values for the symbol that, if applied to its 
trading activity during the prior trading day, would have caused the 
System to deem trading in the symbol to be extraordinarily unstable 
for as close to one percent of that day as possible.
---------------------------------------------------------------------------

    The following is an illustration of the operation of the stability 
protection mechanism. At 11:10:04 a.m., the prevailing Holding Period 
for M-ELOs in symbol XYZ is 1.5 milliseconds. At the same time, the 
NBBO for symbol XYZ updates. The System looks back at the prior three 
seconds of trading in symbol XYZ and finds that during that period, the 
highest observed NBBO midpoint was $10.05, and the lowest was $10.00, 
such that the difference between these two values is a range of $0.05. 
The System then looks back at trading behavior for symbol XYZ during 
the immediately preceding trading day. In doing so, the System 
calculates the value of the threshold that would have caused the symbol 
to be deemed extraordinarily unstable for one percent of the trading 
day; the System determines that this threshold value is a range of 
$0.03. The System then compares the $0.03 threshold to its measurement 
of the prior three seconds of NBBO changes ($0.05), and concludes that 
over these past three seconds, the symbol is extraordinarily unstable. 
Accordingly, the System activates the stability protection mechanism 
and the Holding Period for M-ELOs in symbol XYZ immediately increases 
to 12 milliseconds for a period of 750 milliseconds. However, 5 
milliseconds after the Stability Protection Period commences, the NBBO 
updates again, thus prompting the System to repeat its assessment of 
the stability of the symbol in light of the update. This reassessment 
reveals that the symbol remains unstable, such that a new Stability 
Protection Period of 750 milliseconds begins at that time (overriding 
the pre-existing Period). Over the course of this new Stability 
Protection Period, the NBBO shifts two more times, but each of the 
ensuing reassessments indicate that the NBBO ranges for the symbol have 
fallen below the $0.03 threshold. The Stability Protection Period 
elapses 750 milliseconds after it began with the symbol remaining 
stable. Thus, the Holding Period reverts to 1.5 milliseconds.
    If the Exchange halts trading in a symbol, then upon resumption of 
trading, any new M-ELO or M-ELO+CB in that symbol and any pending M-ELO 
or M-ELO+CB in that symbol with an unexpired Holding Period will be 
subject to a new 12 milliseconds Holding Period (running from the time 
when trading resumes) until the next scheduled Change Event, at which 
point the System may determine to adjust that Holding Period to a 
duration within the range applicable under normal market 
conditions.\27\ If, however, the System determines that extraordinary 
instability in the symbol exists, it will instead determine to activate 
the stability protection mechanism and maintain the duration of the 
Holding Period at 12 milliseconds for another 750 milliseconds. This 
design will help to ensure that M-ELOs and M-ELO+CBs receive added 
protection coming out of halt conditions.\28\
---------------------------------------------------------------------------

    \27\ Prior to commencement of a new 12 millisecond Holding 
Period for a new or pending M-ELO or M-ELO+CB following a Halt, the 
System will first determine whether the M-ELO or M-ELO+CB is or 
remains eligible for execution. That is, the Holding Period will 
commence only if, upon commencement of trading following the Halt, 
the midpoint price for the Order is within the limit set by the 
participant. If not, the System will hold the Order until the 
midpoint falls within the limit set by the participant, at which 
time the 12 millisecond Holding Period will commence.
    \28\ Also as a safeguard, the System will apply a default 
Holding Period of 12 milliseconds to a M-ELO or M-ELO+CB if ever it 
fails to receive a signal during a Change Event as to whether the 
System should adjust or maintain the duration of the prevailing 
Holding Period. The System will continue to apply the default 12 
millisecond Holding Period until the next Change Event where the 
signal is restored and the System is able to act dynamically again.
---------------------------------------------------------------------------

    The Exchange notes that same dynamic process described above will 
also apply to and govern the time periods during which Midpoint Orders 
on the Continuous Book must rest before they will become eligible to 
interact with M-ELO+CBs (provided that participants have opted for 
their Midpoint Orders to interact with M-ELO+CBs). Thus, the same 
Holding Period duration that the System sets for a M-ELO+CB in a symbol 
during Regular Market Hours will also be the length of time that a 
Midpoint Order must rest on the Continuous Book must rest before it may 
interact with a M-ELO+CB.
    Apart from these impacts of Dynamic Holding Periods, M-ELOs and M-
ELO+CBs will continue to behave as they do now in all respects, and as 
set forth in Rules 4702(b)(14) and (15).
    It is important to note that within the parameters discussed herein 
and in the White Paper, the Exchange will continue to re-train Dynamic 
M-ELO and M-ELO+CB on a weekly basis (outside of market hours) so that 
the model will continue to learn from and act upon the basis of more 
recent SIP and M-ELO book data sets, and further improve its 
performance over time. The retraining process should not result in 
dramatic or unpredictable changes to the behavior of Dynamic M-ELO. The 
retraining process will not retrain the model from scratch each week; 
rather, it will retain the model's existing data inputs, knowledge 
base, and objectives--all without alteration. Retraining will result in 
new behaviors only as needed to address new scenarios that the model 
did not confront previously, and even then, only in a manner designed 
to further optimize outcomes, i.e., reduce mark-outs or increase fill 
rates. If the System assesses that a retrained model would be worse 
than the existing model in achieving its objectives, then the System 
will continue to use the existing model and discard the retrained 
model. This retraining process is a standard and accepted practice for 
use of deep learning models; it helps to ensure that deep learning 
models not only work well, but that they continue to work well in 
dynamic circumstances.\29\
---------------------------------------------------------------------------

    \29\ During periods where the model is not undergoing 
retraining, the System will behave predictably from day to day, such 
that its decisions when presented with given set of facts and 
circumstances in a given security on day 1 should be the same as 
they would be on day 2.
---------------------------------------------------------------------------

    The Exchange will not modify the underlying structure of Dynamic M-
ELO and M-ELO+CB without first obtaining the Commission's approval to 
do so, including modifications to the data elements the model considers 
in making decisions about Holding Period durations, the conditions 
under which the model may adjust the duration of Holding Periods, the 
frequency with which the model my adjust the Holding Periods, the range 
of Holding Period durations available to M-ELOs and M-ELO+CBs, the 
increments by which Holding Periods may change at any given Change 
Event, and the procedures for triggering, maintaining, and ending 12 
millisecond Holding Periods during times of extraordinary 
instability.\30\ Although the Exchange will seek Commission approval 
prior to changing any of the data elements that the model considers, 
the Exchange will not seek Commission approval prior to retraining

[[Page 62855]]

the model to adjust the weighting it applies to those data elements.
---------------------------------------------------------------------------

    \30\ In addition to the proposed changes described above, the 
Exchange proposes to delete an extraneous reference in Rule 
4702(b)(15) to M-ELO+CB being eligible to execute against a Midpoint 
Order on the Continuous Book if the Continuous Book order has the 
``Midpoint'' Trade Now Attribute enabled. In a prior filing, the 
Exchange folded the concept of ``Midpoint Trade Now'' into the 
general ``Trade Now'' Attribute. See Securities Exchange Act Release 
No. 34-92180 (June 15, 2021), 86 FR 33420 (June 24, 2021)(SR-NASDAQ-
2021-044).
---------------------------------------------------------------------------

    To aid investors in understanding and evaluating Dynamic M-ELO, 
Nasdaq will continue to publish weekly and monthly transparency 
statistics on Nasdaqtrader.com, as it does now, about the performance 
of its M-ELOs and M-ELO+CBs, including statistics listing the weekly 
numbers of shares and trades in M-ELOs by symbol, weekly aggregated M-
ELO share and trade data, and monthly aggregated block data.\31\ Nasdaq 
also will continue to disclose monthly data on Nasdaq.com, as it does 
now (the M-ELO Monthly Report), about M-ELO and M-ELO+CB mark-outs 
(quote stability by time horizon) and fill rates.\32\ Moreover, Nasdaq 
will add statistics to the M-ELO Monthly Report about how frequently, 
on average, the System changes Holding Period durations for the top 
decile, median, and bottom decile of symbols, as measured by monthly M-
ELO and M-ELO+CB trading volumes. Nasdaq will retain copies of each 
historical iteration of its models as part of its books and records, 
and make them available to the Commission upon request, should it wish 
to examine them to understand how the model changes over time. 
Furthermore, Nasdaq will publish an equity trader alert in advance of 
deploying a retrained version of Dynamic M-ELO whenever Nasdaq has 
reason to anticipate that the retrained version will produce results 
that differ materially from the prior version, i.e., a projected change 
in mark-outs or fill-rates of 10% or more in either direction.
---------------------------------------------------------------------------

    \31\ See https://www.nasdaqtrader.com/Trader.aspx?id=MELOSymbolData.
    \32\ See, e.g., https://www.nasdaq.com/docs/M-ELO-Monthly-Report. Nasdaq understands that current users of M-ELO and M-ELO 
independently monitor the performance of these Order Types. Nasdaq 
often receives feedback from such users about M-ELO and M-ELO+CB 
performance, which Nasdaq then factors into decisions about 
improvements and enhancements. Nasdaq expects that this feedback 
loop will continue after implementation of Dynamic M-ELO.
---------------------------------------------------------------------------

    The Exchange acknowledges that systems necessary to implement 
Dynamic M-ELO, including the systems proposed that include model 
development and retraining processes, are ``SCI Systems'' within the 
meaning of Regulation Systems Compliance and Integrity (``Reg. 
SCI''),\33\ and that the Exchange, as an SCI Entity, remains 
responsible for compliance with all requirements of Reg. SCI, 
including, without limitation, to have policies and procedures 
reasonably designed to ensure that its SCI Systems operate in a manner 
that complies with the Act and the rules and regulations thereunder and 
Exchange's rules and governing documents, among them a plan for 
assessments of the functionality of SCI Systems designed to detect 
systems compliance issues, including by responsible SCI personnel and 
by personnel familiar with applicable provisions of the Act and the 
rules and regulations thereunder and Exchange's rules and governing 
documents.
---------------------------------------------------------------------------

    \33\ 17 CFR 242.1000 et seq. As set forth in Reg. SCI, the term 
``SCI Systems'' means ``means all computer, network, electronic, 
technical, automated, or similar systems of, or operated by or on 
behalf of, an SCI entity that, with respect to securities, directly 
support trading, clearance and settlement, order routing, market 
data, market regulation, or market surveillance.'' Id. at 242.1000. 
An ``SCI Entity'' means ``an SCI self-regulatory organization, SCI 
alternative trading system, plan processor, exempt clearing agency 
subject to ARP, or SCI competing consolidator.'' Id.
---------------------------------------------------------------------------

Implementation
    The Exchange intends to make the proposed change effective for M-
ELOs and M-ELO+CBs in the Second or Third Quarter of 2023, but that 
time frame is subject to change. The Exchange will publish a Trader 
Alert in advance of making the proposed change effective.
2. Statutory Basis
    The Exchange believes that its proposal is consistent with Section 
6(b) of the Act,\34\ in general, and furthers the objectives of Section 
6(b)(5) of the Act,\35\ in particular, in that it is designed to 
promote just and equitable principles of trade, to remove impediments 
to and perfect the mechanism of a free and open market and a national 
market system, and, in general to protect investors and the public 
interest, by allowing for more widespread use of M-ELOs and M-ELO+CBs.
---------------------------------------------------------------------------

    \34\ 15 U.S.C. 78f(b).
    \35\ 15 U.S.C. 78f(b)(5).
---------------------------------------------------------------------------

    When the Commission approved the M-ELO and the M-ELO+CB, it 
determined that these Order Types are consistent with the Act because 
they ``could create additional and more efficient trading opportunities 
on the Exchange for investors with longer investment time horizons, 
including institutional investors, and could provide these investors 
with an ability to limit the information leakage and the market impact 
that could result from their orders.'' \36\ Nothing about the 
Exchange's proposal should cause the Commission to revisit or rethink 
this determination. Indeed, the proposal will not alter the fundamental 
design of these Order Types, the manner in which they operate, or their 
effects.
---------------------------------------------------------------------------

    \36\ M-ELO Approval Order, supra 83 FR at 10938-39; M-ELO+CB 
Approval Order, supra, 84 FR at 48980.
---------------------------------------------------------------------------

    Even with Dynamic M-ELO Holding Periods, M-ELOs and M-ELO+CBs will 
continue to provide their users with protection against information 
leakage and adverse selection--and they will do so at levels which are 
substantially undiminished from that which they provide now.\37\
---------------------------------------------------------------------------

    \37\ See note 6, supra.
---------------------------------------------------------------------------

    At the same time, however, the proposal will benefit market 
participants and investors by reducing the opportunity costs of 
utilizing M-ELOs and M-ELO+CBs. The proposal, in other words, will re-
calibrate the lengths of the Holding Periods so that M-ELOs and M-
ELO+CBs will operate in the ``Goldilocks'' zone--their Holding Periods 
will not be so short as to render them unable to provide meaningful 
protections against information leakage and adverse selection, but the 
Holding Periods also will not be too long so as to cause participants 
and investors to miss out on favorable execution opportunities. Nasdaq 
believes the proposal will render M-ELOs and M-ELO+CBs more useful and 
attractive to market participants and investors, and this increased 
utility and attractiveness, in turn, will spur an increase in M-ELO and 
M-ELO+CB use cases on the Exchange, both from new and existing users of 
M-ELOs and M-ELO+CBs. Ultimately, the proposal should enhance market 
quality by increasing opportunities for midpoint executions on the 
Exchange.
    As Nasdaq explained above, the Proposal will operate within strict, 
well-defined, and transparent parameters. Although it will undergo 
weekly retraining (outside of market hours),\38\ such retraining will 
aim to improve the performance of the model in achieving its twin 
objectives; retraining will not alter the inputs, objectives, or basic 
design parameters of Dynamic M-ELO without prior Commission 
approval.\39\ Moreover, the Exchange will not deploy a retrained model 
if it fails to achieve performance improvements. To aid

[[Page 62856]]

investors in evaluating Dynamic M-ELO, the Exchange will publish 
statistics about its performance, including as to mark-outs and fill 
rates, as well as statistics about how frequently the System changes 
Holding Period durations. To further facilitate accountability, the 
Exchange will retain each historical iteration of its model as part of 
its books and records, and make such information available to the 
Commission, upon request. The Exchange will also publish equity trader 
alerts whenever retraining will result in a performance change of 10% 
or more.
---------------------------------------------------------------------------

    \38\ To be clear, performance statistics for Dynamic M-ELO cited 
herein and in the White Paper are based upon data derived from 
weekly, not daily retrainings.
    \39\ As discussed above, Nasdaq will not seek Commission 
approval prior to allowing the model, as part of its re-training 
process, to vary the weighting of the data elements it ingests. 
Nasdaq believes this is appropriate because such variance will only 
occur to the extent that it will improve the model's performance 
with respect to pre-defined objectives. Nasdaq will alert traders if 
the retraining process would result in substantial performance 
changes, and it will also publish statistics to help participants to 
assess performance themselves. Moreover, Nasdaq will retain 
historical iterations of its models for the Commission's review, 
should it wish to examine how these models have changed over time.
---------------------------------------------------------------------------

    Nasdaq notes that the twin objectives it prescribes for the model 
involve the absolute values of mark-outs and fill rates; they are not 
designed to further the performance of any participant or any category 
of participant. Furthermore, Nasdaq performed internal tests of its AI 
model to detect indications of harmful bias in its performance results, 
and such tests concluded that no such indications exist. That is, the 
Exchange reviewed the impact on fill rates and mark-outs of Dynamic M-
ELO, as compared to the ``static'' M-ELO, for those firms that 
accounted for more than 95% of M-ELO activity on the Exchange during Q1 
2022.\40\ The Exchange analyzed results both in an absolute and a 
relative sense. Testing revealed that all participants experienced at 
least some improvements in fill rates and mark-outs when using Dynamic 
M-ELO versus static M-ELO, with the volume-weighted average improvement 
being aligned with the results expressed in the White Paper. We 
detected no material variations that might suggest that a particular 
participant or category of participant (i.e., nature of firm; size of 
firm) benefitted from Dynamic M-ELO functionality to an extent that was 
unreasonably disproportionate to the benefits that other participants 
experienced. Thus, Nasdaq believes the model is objective, is designed 
to, and does avoid bias and discrimination.\41\
---------------------------------------------------------------------------

    \40\ Beyond this grouping of participants, the activity levels 
of other individual M-ELO participants were so small as to be 
insignificant. In many cases, these participants entered only a 
handful of M-ELOs during the study period. As such, the Exchange 
believes it is reasonable to exclude such participants from its 
analysis to avoid their data distorting the results.
    \41\ The Exchange will review its AI model periodically to 
affirm that it continues to perform in accordance with the 
Exchange's rules and has not introduced any harmful bias in favor of 
or against any participant or category of participants.
---------------------------------------------------------------------------

    The Exchange notes that use of Dynamic M-ELOs and M-ELO+CBs remains 
voluntary for all market participants. Accordingly, if any market 
participant feels that the dynamic Holding Periods are still too long 
or too short or because competing venues offer more attractive delay 
mechanisms, then the participants are free to pursue other trading 
strategies or utilize other trading venues. They need not utilize 
Dynamic M-ELOs or M-ELO+CBs.
    Furthermore, the design of Dynamic-MELO would constitute an 
``established, non-discretionary'' method that is consistent with the 
definition of an exchange, as set forth in SEC Rule 3b-16.\42\ The 
Commission stated as follows when it adopted Rule 3b-16:
---------------------------------------------------------------------------

    \42\ See 17 CFR 240.3b-16(a)(2) (``(a) An organization, 
association, or group of persons shall be considered to constitute, 
maintain, or provide `a market place or facilities for bringing 
together purchasers and sellers of securities or for otherwise 
performing with respect to securities the functions commonly 
performed by a stock exchange,' as those terms are used in section 
3(a)(1) of the Act, (15 U.S.C. 78c(a)(1)), if such organization, 
association, or group of persons: (1) Brings together the orders for 
securities of multiple buyers and sellers; and (2) Uses established, 
non-discretionary methods (whether by providing a trading facility 
or by setting rules) under which such orders interact with each 
other, and the buyers and sellers entering such orders agree to the 
terms of a trade.'').

    A system uses established non-discretionary methods either by 
providing a trading facility or by setting rules governing trading 
among subscribers. The Commission intends for ``established, non-
discretionary methods'' to include any methods that dictate the 
terms of trading among the multiple buyers and sellers entering 
orders into the system. Such methods include those that set 
procedures or priorities under which open terms of a trade may be 
determined. For example, traditional exchanges' rules of priority, 
parity, and precedence are ``established non-discretionary 
methods,'' as are the trading algorithms of electronic systems. 
Similarly, systems that determine the trading price at some 
designated future date on the basis of pre-established criteria 
(such as the weighted average trading price for the security on the 
specified date in a specified market or markets) are using 
established, non-discretionary methods.\43\
---------------------------------------------------------------------------

    \43\ See Securities Exchange Act Release No. 40760 (December 8, 
1998), 63 FR 70844, 70850 (December 22, 1998).

    Nothing in the Reg. ATS Adopting Release or in any of its 
illustrative examples suggests that Dynamic M-ELO would constitute an 
exercise of discretionary behavior. Dynamic M-ELO will handle and 
execute Orders according to published, pre-determined rules that are 
disclosed to the public and which provide reasonable notice of how the 
Order Type will behave.\44\ To the extent that the design of the System 
permits variation in the Holding Periods for such Orders, it does so by 
design. The range of potential variations, the objectives that such 
variations are intended to achieve, and the factors that determine when 
such variations may occur are also predetermined and set forth in the 
Exchange's Rules or otherwise disclosed to the public. The mere fact 
that the System may apply different weights over time to the factors it 
uses to determine whether and by how much to vary a Holding Period does 
not mean that the System will act with discretion in the same sense 
that a human being could be said to be exercise independent judgment 
when deciding whether and how to handle an order.\45\ Even when the 
System makes decisions about changing the Holding Periods, the System 
will operate pursuant to a mathematical algorithm from which it cannot 
deviate--an algorithm that is programmed to achieve pre-defined and 
pre-disclosed objectives.\46\
---------------------------------------------------------------------------

    \44\ See id. at 70900 (``an essential indication of the non-
discretionary status of rules and procedures is that those rules and 
procedures are communicated to the systems users'' and ``[t]hus, 
participants have an expectation regarding the manner of execution--
that is, if an order is entered, it will be executed in accordance 
with those procedures and not at the discretion of a counterparty or 
intermediary.'').
    \45\ Cf. id. at 70851 (explaining that a traditional block 
trading desk is an example of a system that does not use 
established, non-discretionary methods because the operators of such 
desks do not act according to fixed procedures known to their 
customers, but instead shop orders around for potential 
counterparties and make their own determinations as to whether and 
how to execute block orders, including by sometimes deciding to take 
a proprietary position in part of the block order).
    \46\ See id. at 80755 (describing an example of a system that 
would be non-discretionary in nature: ``System I permits 
participants to enter a range of ranked contingent buy and sell 
orders at which they are willing to trade securities. These orders 
are matched based on a mathematical algorithm whose priorities are 
designed to achieve the participants' objectives. System I does not 
display orders to any participants. System I is included under Rule 
3b-16.''); see also Securities Exchange Act Release No. 34-89686 
(August 20, 2020), 85 FR 54438, at 54445, n.92 (September 1, 2020) 
(Order approving SR-IEX-2019-15) (rejecting argument that IEX's D-
Limit order time is an exercise of discretion because ``D-Limit 
orders will not allow IEX to exercise any discretion on any 
particular order by deviating from the CQI and D-Limit 
functionality, which is hardcoded in the IEX rulebook.''; Securities 
Exchange Act Release No. 34-78101 (June 17, 2016), 81 FR 41141, at 
41153(June 17, 2016) (Order approving IEX Form 1 and D-Peg Order 
Type) (``the Commission does not believe that the hardcoded 
conditionality of the IEX proposed ``discretionary'' peg order type 
provides IEX with actual discretion or the ability to exercise 
individualized judgment when executing an order. Rather, if IEX's 
fixed formula determines the quote to be stable, the discretionary 
peg order can execute up to the midpoint; if it does not deem the 
quote to be stable, then it will hold the order to its pegged price. 
As such, IEX would not exercise discretion over the routing and 
execution of a resting order''). Nasdaq does not believe that it is 
necessary to codify its mathematical formula for Dynamic M-ELO in 
its Rules because Nasdaq has disclosed sufficient information in its 
Rules and in its filing to inform the public as to the possible and 
expected behaviors associated with Dynamic M-ELO, as well as a means 
for the Commission and/or investors to verify whether Dynamic M-ELO 
is performing appropriately. Much as the Commission does not require 
an exchange to codify the source code it uses to effectuate other 
behaviors or actions that it explains in its Rules, including the 
behaviors of other complex Order Types, there is no basis to require 
codification of the Dynamic M-ELO formula in this instance.

---------------------------------------------------------------------------

[[Page 62857]]

    The Exchange notes that it will continue to conduct real-time 
surveillance to monitor the use of M-ELOs and M-ELO+CBs to ensure that 
such usage remains appropriately tied to the intent of the Order Types. 
If, as a result of such surveillance, the Exchange determines that the 
Dynamic M-ELO Holding Periods do not serve their intended purposes, or 
adversely impact market quality, then the Exchange will seek to make 
further re-calibrations.
    Nasdaq does not believe that the design of Dynamic M-ELO lends 
itself to potential manipulation by a single participant or a small 
group of participants because the System makes determinations regarding 
Holding Periods based upon prevailing market-wide conditions for a 
given symbol, rather than the behaviors of particular participants with 
respect to that symbol, or the activity of participants in M-ELOs 
involving that symbol. Manipulation of the System also would be 
difficult to accomplish given the large number of variables that factor 
into the System's decisions to change Holding Periods during Change 
Events, as well as the different weights that apply to each such 
factor, which as described above, the System may vary over time. Any 
benefits that a participant might derive from manipulating the duration 
of Holding Periods would likely be small and outweighed significantly 
by the difficulty and cost of affecting such manipulation. 
Nevertheless, the Exchange will surveil for indications of manipulation 
and act accordingly if it detects such indications.

B. Self-Regulatory Organization's Statement on Burden on Competition

    The Exchange does not believe that the proposed rule change will 
impose any burden on competition not necessary or appropriate in 
furtherance of the purposes of the Act. To the contrary, the Exchange 
believes that this proposal will promote the competitiveness of the 
Exchange by rendering its M-ELO and M-ELO+CB Order Types more 
attractive to participants.
    The Exchange adopted the M-ELO and M-ELO+CB as pro-competitive 
measures intended to increase participation on the Exchange by allowing 
certain market participants that may currently be underserved on 
regulated exchanges to compete based on elements other than speed. The 
proposed change continues to achieve this purpose. With Dynamic M-ELO 
Holding Periods, both M-ELOs and M-ELO+CBs will afford their users with 
a level of protection from information leakage and adverse selection 
that is better from what is achievable at present.\47\ At the same 
time, the Dynamic Holding Periods will increase opportunities to 
interact with other like-minded investors with longer time horizons 
while also lowering the opportunity costs for participants that utilize 
M-ELOs and M-ELO+CBs, particularly for securities that trade within the 
``Goldilocks'' zone. In sum, the proposed changes will not burden 
competition, but instead may promote competition for liquidity in M-
ELOs and M-ELO+CBs by broadening the circumstances in which market 
participants may find such Orders to be useful. With the proposed 
changes, market participants will be more likely to determine that the 
benefits of entering M-ELOs and M-ELO+CBs outweigh the risks of doing 
so.
---------------------------------------------------------------------------

    \47\ See White Paper, supra.
---------------------------------------------------------------------------

    The proposed change will not place a burden on competition among 
market venues, as any market may adopt an order type that operates 
similarly to a M-ELO or a M-ELO+CB with Dynamic M-ELO Holding Periods.

C. Self-Regulatory Organization's Statement on Comments on the Proposed 
Rule Change Received From Members, Participants, or Others

    No written comments were either solicited or received.

IV. Discussion and Commission Findings

    After careful review, the Commission finds that the Exchange's 
proposal to adopt Dynamic M-ELO is consistent with the requirements of 
the Act and the rules and regulations thereunder applicable to a 
national securities exchange.\48\ In particular, the Commission finds 
that the proposed rule change, as modified by Amendment No. 2, is 
consistent with Sections 6(b)(5) and 6(b)(8) of the Act.\49\ Section 
6(b)(5) of the Act requires that the rules of a national securities 
exchange be designed, among other things, to prevent fraudulent and 
manipulative acts and practices, to promote just and equitable 
principles of trade, and, in general, to protect investors and the 
public interest, and not be designed to permit unfair discrimination 
between customers, issuers, brokers, or dealers.\50\ Section 6(b)(8) of 
the Act requires that the rules of a national securities exchange not 
impose any burden on competition that is not necessary or appropriate 
in furtherance of the purposes of the Act.\51\
---------------------------------------------------------------------------

    \48\ In approving this proposed rule change, the Commission has 
considered the proposed rule's impact on efficiency, competition, 
and capital formation. See 15 U.S.C. 78c(f). One commenter 
questioned whether Nasdaq's pending patent applications for the 
systems it will use to operate Dynamic M-ELO imposes an unnecessary 
or inappropriate burden on competition. See Letter from R.T. 
Leuchtkafer, dated January 21, 2023 (``Leuchtkafer Letter 1''), at 
3. The Commission does not believe that the sole fact that Nasdaq 
has a pending patent application for the technology it has developed 
to operate the Dynamic M-ELO is indicative that the operation of 
Dynamic M-ELO on the Exchange would place an inappropriate burden on 
competition. As explained below, Nasdaq has provided sufficient 
public disclosure and analysis to explain how Dynamic M-ELO will 
operate.
    \49\ In addition to providing a statutory analysis in its 
filing, Nasdaq also acknowledges, above in Amendment No. 2, that the 
systems it will use to implement Dynamic M-ELO, including the 
Exchange's model development and retraining processes, are SCI 
systems under Regulation SCI, see 17 CFR 242.1000 et seq., and thus, 
it will be responsible for compliance with Regulation SCI with 
respect to Dynamic M-ELO, including having appropriate policies and 
procedures. See supra note 33 and accompanying text.
    \50\ 15 U.S.C. 78f(b)(5).
    \51\ 15 U.S.C. 78f(b)(8).
---------------------------------------------------------------------------

Nasdaq's Obligation To Sufficiently Explain Its Proposed Rule Change

    The burden to demonstrate that a proposed rule change is consistent 
with the Act and rules and regulations thereunder is on the self-
regulatory organization (``SRO'') proposing a rule change.\52\ Each 
proposed SRO rule change must be ``accompanied by a concise general 
statement of the basis and purpose of such proposed rule change.'' \53\ 
As described in more detail below, several commenters argued that the 
proposal did not provide sufficient information with respect to the 
operation of Dynamic M-ELO, or that the information provided was not 
``clear and comprehensible,'' as required by Form 19b-4. For the 
reasons articulated below, the Commission believes that Nasdaq has 
provided clear and comprehensible information on the overall operation 
of Dynamic M-ELO and the role of the machine-learning model and 
demonstrated that the proposal is consistent with the Act. Several 
related comments addressed this issue; these comments and Nasdaq's 
responses are discussed below, followed by the Commission's analysis.
---------------------------------------------------------------------------

    \52\ See 17 CFR 201.700(b)(3).
    \53\ 15 U.S.C 78s(b)(1).
---------------------------------------------------------------------------

    One commenter stated that the initial filing would establish ``a 
dangerously vague standard for describing how exchange-hosted complex 
algorithmic

[[Page 62858]]

order types operate.'' \54\ In response to public comment, Nasdaq added 
more details describing the operation of Dynamic M-ELO to both the 
filing and public record since this proposed rule change was initially 
submitted to the Commission, and Nasdaq also provided additional legal 
analysis to support Dynamic M-ELO's consistency with the Act. Prior to 
the filing of Amendment No. 1, a commenter stated that although 
``Nasdaq shared some of the 142 features of their formula,'' Nasdaq 
should reveal all of these features so that prospective users may 
evaluate how the model works.\55\ Similarly, another commenter stated 
that the public cannot provide meaningful comment on the proposal 
without knowing all categories and parameters of the proposed Dynamic 
M-ELO.\56\ In its response to these comments, the Exchange, among other 
things, provided the specific 142 data elements that will be weighed by 
the machine-learning model as both an appendix to its first letter in 
response to comments,\57\ and as Exhibit 3B to its Amendment No. 1 
filing.
---------------------------------------------------------------------------

    \54\ See Leuchtkafer Letter 1, supra note 48, at 1-2. See also 
Letter from Joseph Saluzzi, Partner, Themis LLC, dated January 25, 
2023, at 3 (``Themis Letter'') (questioning whether the complexity 
of Dynamic M-ELO is necessary).
    \55\ See Themis Letter, supra note 54, at 2.
    \56\ See Leuchtkafer Letter 1, supra note 48, at 1.
    \57\ See Letter from Brett Kitt, Associate Vice President and 
Principal Associate General Counsel, Nasdaq, Inc., dated March 9, 
2023, at Appendix A (``Nasdaq First Response to Comments'').
---------------------------------------------------------------------------

    In response to the Exchange's disclosures in Exhibit 3B of 
Amendment No. 1, one of these commenters stated that the list of data 
elements was not ``clear and comprehensible'' as is required by the 
Form 19-4, but rather ``vague, confusing, and perfunctory.'' \58\ This 
commenter also stated that the disclosed data elements included 
unexplained terms (e.g., ``baseline simulated,'' ``action simulated,'' 
and ``synthetic mark-out'').\59\ In a subsequent comment letter, this 
commenter reiterated these points; the commenter specified that the 
commenter's concern is that Nasdaq's rule text does not disclose 
information about its methods for assessing market conditions and that 
``Nasdaq should carefully detail its methods in its rulebook, just like 
other exchanges have done, and Nasdaq should also thoroughly disclose 
its methods in its filing text.'' \60\ In its second response to 
comments \61\ and the revised Exhibit 3b to Amendment No. 2, Nasdaq 
expanded and ``simplified'' the explanation of these 142 data elements. 
The Exchange also added to the proposed rule text a definition of the 
term ``proprietary assessment of market conditions'' to explain how the 
machine-learning model will evaluate those 142 data elements.\62\
---------------------------------------------------------------------------

    \58\ See Letter from R.T. Leuchtkafer, dated May 2, 2023, at 8-9 
(``Leuchtkafer Letter 2'').
    \59\ See id.
    \60\ See Letter from R.T. Leuchtkafer dated May 30, 2023, at 3-
5; 8-9 (``Leuchtkafer Letter 3''). This commenter also cites to the 
rules governing the Crumbling Quote Indicator and D-Limit order type 
on the Investors Exchange (``IEX''), as well as language from the 
Commission's approval order for the D-Limit order type. See id. at 
4. The commenter notes the level of detail with regard to how and 
when the D-Limit order type exercises its discretionary price-
sliding that is set forth in the IEX Rulebook. See id. at 4; see 
also Themis Letter, supra note 54, at 2 (``Another exchange, IEX, 
operates a smart logic called CQI (Crumbling Quote Indicator) which 
aims to protect orders from being adversely selected. IEX has 
published detailed notes on how the CQI is calculated.''). Each 
proposal must be evaluated based on the specific facts and 
circumstances before the Commission. In this case, the Commission is 
only reviewing the proposed operation of Dynamic M-ELO and its 
machine-learning model. Accordingly, the level of detail provided in 
the IEX Rulebook for the D-Limit order type and Crumbling Quote 
Indicator--or the rulebooks for order types on other exchanges--does 
not determine whether Nasdaq has met its burden in this proposal.
    \61\ See Letter from Brett Kitt, Associate Vice President and 
Principal Associate General Counsel, Nasdaq, Inc., dated May 18, 
2023 at Appendix A (``Nasdaq Second Response to Comments'').
    \62\ See supra note 25.
---------------------------------------------------------------------------

    Furthermore, Nasdaq attached, as Exhibit 3A to its proposed rule 
change, the White Paper written by its AI Core Development Team that 
explains, among other things, how Dynamic M-ELO's machine-learning 
functions were developed and tested. The White Paper includes a general 
discussion of the type of model implemented in the proposed system, in 
this case a reinforcement learning model,\63\ as well as citations to 
academic research behind the Double Deep Q-Network algorithm that is 
the basis for the algorithm used in Nasdaq's model.\64\ The White Paper 
also describes the ways in which Nasdaq's implementation of the 
proposed model differs from the model and training in the academic 
research, providing both an English summary and a pseudocode 
description of differences in model training implemented by Nasdaq.\65\
---------------------------------------------------------------------------

    \63\ See White Paper Section 3.1.
    \64\ See White Paper Section 4.1.
    \65\ See White Paper, Section 7.2.
---------------------------------------------------------------------------

    One of the commenters stated that the White Paper is not easily 
understood by most market participants and that referencing the White 
Paper in the filing is an ``unacceptable substitute'' for a ``plain 
English'' explanation of the proposal in Form 19b-4.\66\ In response, 
Nasdaq explained that it drafted the filing to provide a general 
understanding of Dynamic M-ELO and how it will behave, and the more 
detailed information and explanation in the White Paper are meant to 
support the filing.\67\
---------------------------------------------------------------------------

    \66\ See Leuchtkafer Letter 2, supra note 58, at 3-4; 
Leuchtkafer Letter 3, supra note 60, at 4-5; Letter from R.T. 
Leuchtkafer, dated August 11, 2023, at 8-10 (``Leuchtkafer Letter 
4'').
    \67\ See Nasdaq Second Response to Comments, supra note 61, at 
3-5.
---------------------------------------------------------------------------

    The Commission agrees with comments and the Exchange that there is 
an extent to which the proposed changes will introduce an unavoidable 
degree of uncertainty with respect to the use of these order types. The 
deep reinforcement learning model that will determine the dynamic 
holding periods for each symbol for M-ELO and M-ELO+CB orders will be 
implemented through established, non-discretionary methods,\68\ but it 
is so complex that its complete details are, for most intents and 
purposes, not readily intelligible, and it would be immensely difficult 
for the Exchange or any market participant to precisely predict the 
holding periods that will be generated by the model for any given 
symbol at any particular time. Nevertheless, as further discussed 
below, the Commission believes that the Exchange has provided 
information sufficient for the Commission and public to understand the 
design, operation, and limits of the proposed changes to these order 
types, and the role of the machine-learning model therein.
---------------------------------------------------------------------------

    \68\ See infra notes 91-92 and accompanying text.
---------------------------------------------------------------------------

    While the holding periods under the proposal would be dynamic, 
Nasdaq has precisely articulated both the nature of changes that would 
be permissible under the proposal, and the limits to those changes. 
Nasdaq described when changes might occur (every thirty seconds 
throughout the trading day), the initial default holding period for all 
symbols (1.25 milliseconds), the permissible increments by which a 
holding period might change in each symbol (0.25 or 0.50 milliseconds), 
and the outer bounds of permissible holding period lengths (0.25 
milliseconds at the short end, and 2.50 milliseconds at the long end). 
Nasdaq also described the conditions of ``extraordinary instability'' 
in a symbol when these holding periods would not apply, and when the 
holding periods would be overridden by the proposed ``stability 
protection mechanism'' (with a holding period of 12 milliseconds for at 
least 750 milliseconds). The Commission believes that these details 
provide sufficient information to understand the range of potential 
holding periods that may be applied when M-ELO or M-ELO+CB orders are 
entered or resting on the order book, the changes that may occur, and 
the limits to those changes.

[[Page 62859]]

    Nasdaq has also described the role of the machine-learning model in 
its proposal. The model will determine whether, by which increment, and 
in which direction to adjust the holding period for each symbol 
throughout the trading day.\69\ In its Form 19b-4, White Paper, and 
response letters, Nasdaq described the goals towards which the model is 
optimized: reducing mark-outs and increasing fill rates.\70\ Nasdaq's 
White Paper includes a detailed discussion of model choice, 
development, and training, including citations to relevant other 
research.\71\ Nasdaq also provided several iterations of a list of data 
elements that the model will ingest and use, including a glossary 
defining terms used in the descriptions.\72\ Nasdaq affirmed that in 
operation during market hours, the data used would be calculated based 
on intraday market data.\73\ One version of these lists included 
Nasdaq's estimates of the tendencies of data elements to affect model 
outcomes.\74\ Nasdaq's White Paper also included an ``explainability 
study'' that assessed both the effects of individual data elements on 
model performance and the effects of interactions between individual 
data elements.\75\ Across its filing and incorporated exhibits, aspects 
of the model's operations and design are described in different formats 
and with different levels of specificity--for example, the filing and 
exhibits include ``plain English'' descriptions, mathematical 
definitions, and pseudocode. Together, this set of information allows 
the Commission to understand the type of decision the model will 
implement, the goals the model aims to achieve, which model type is 
implemented and how it was developed, the range of data types and data 
sources used by the model, and estimates of the manner in which those 
data may affect model outcomes.
---------------------------------------------------------------------------

    \69\ See supra notes 16-24 and accompanying text.
    \70\ See, e.g., Form 19b-4 at 9, White Paper Section 5, and 
Nasdaq First Response to Comments at 2. In its White Paper, Nasdaq 
provides mathematical definitions of fill rate for a period of time 
and mark-out by trade (White Paper at 5, Equations 1 and 2), as well 
as of the assessment made by the agent in the model's reinforcement 
learning process (White Paper at 11, Equation 3).
    \71\ See White Paper Sections 3-5.
    \72\ See Exhibit 3B. As described below, these data elements are 
also those used in training and retraining the model.
    \73\ See Letter from Brett Kitt, Associate Vice President and 
Principal Associate General Counsel, Nasdaq, Inc., dated September 
6, 2023 (``Nasdaq Third Response to Comments'').
    \74\ Nasdaq affirmed that, while this information was not 
included in all versions of the list of data elements, it remains 
accurate and valid. See id. at 3.
    \75\ See White Paper Section 5.3.
---------------------------------------------------------------------------

    Nasdaq also explains how and when the machine-learning model will 
be retrained. Nasdaq will retrain the model weekly, outside of market 
hours. Retraining will incorporate market data obtained during the week 
from the equity consolidated data feeds and M-ELO order book. A 
retrained model will only be promoted to production if it improves upon 
the model objectives compared to the prevailing model.\76\ Furthermore, 
the Exchange explained that the machine-learning model is consistent in 
its behavior from day-to-day during periods when it is not undergoing 
retraining, ``such that its decisions when presented with given set of 
facts and circumstances in a given security on day 1 should be the same 
as they would be on day 2.'' \77\ The Exchange also stated in its 
initial response to comments that ``[e]ven after the system undergoes 
retraining, which will occur on a weekly basis (and not during market 
hours), system behavior should not change dramatically or in unexpected 
ways from week-to-week.'' \78\ As noted above as well, the Exchange 
also represents that outside of set retraining periods, ``the System 
will operate pursuant to a mathematical algorithm from which it cannot 
deviate--an algorithm that is programmed to achieve pre-defined and 
pre-disclosed objectives.'' \79\ Nasdaq also will publish equity trader 
alerts when it anticipates that a model update may change mark-outs or 
fill rate by 10% or more in either direction.\80\ By including this set 
of information, Nasdaq has provided the Commission and public with 
information that allows them to understand how frequently the model 
will be retrained, the data used for retraining, and the criteria that 
will be used to determine whether to update the production model based 
on retraining. This information allows the Commission to understand 
when the proposed model may change and when it will remain constant, 
the circumstances under which a change would be implemented, and 
circumstances under which the public will receive notice of significant 
changes in the model's anticipated outcomes.
---------------------------------------------------------------------------

    \76\ For example, Nasdaq affirmatively states that if ``a 
retrained model would be worse than the existing model in achieving 
its objectives, then the System will continue to use the existing 
model and discard the retrained model.'' See Section III.A.1., 
supra.
    \77\ See Nasdaq First Response to Comments, supra note 57, at 2-
3. See also supra note 30.
    \78\ See Nasdaq First Response to Comments, supra note 57, at 2-
3. A commenter also noted that it was initially unclear when and how 
frequently the machine-learning model would retrain, stating that 
the White Paper set forth an analysis based on daily retraining, but 
the rule filing proposes weekly retraining. See Leuchtkafer Letter 
2, supra note 58, at 4; Leuchtkafer Letter 3, supra note 60, at 2; 
Leuchtkafer Letter 4, supra note 66, at 9. In Amendment No. 2, the 
Exchange affirmatively represents that ``the performance statistics 
for Dynamic M-ELO cited herein and in the White Paper are based upon 
data derived from weekly, not daily retrainings.'' See supra note 
38.
    \79\ See supra note 46 and accompanying text.
    \80\ See, e.g., Amendment No. 2 at 19-20.
---------------------------------------------------------------------------

    In addition, a commenter stated their belief that the Exchange's 
proposal for Dynamic M-ELO would result in the exercise of discretion 
by a national securities exchange because the machine-learning model's 
decisions would vary over time based on the following: (1) varying 
parameter values; and (2) results of retraining cycles.\81\ The 
commenter stated that by making it possible for Dynamic M-ELO to behave 
differently when confronted by the same market conditions before and 
after the model is retrained, Nasdaq's model would be exercising 
discretion that is more akin to a broker than an exchange.\82\
---------------------------------------------------------------------------

    \81\ See Leuchtkafer Letter 1, supra note 48, at 2; Leuchtkafer 
Letter 2, supra note 58, at 4-6; Leuchtkafer Letter 3, supra note 
60, at 5-7; Leuchtkafer Letter 4, supra note 66, at 4-8.
    \82\ See Leuchtkafer Letter 1, supra note 48, at 2. See also 
Leuchtkafer Letter 3, supra note 60, at 5-7. For example, the 
commenter believes that Dynamic M-ELO will ``exercise individualized 
judgment'' such that it can set a different time-in-force for the 
very same order presented in the very same market conditions on, for 
example, August 21 than it set on May 15, depending on the system's 
undisclosed individualized judgments of market conditions and 
participant behavior from even days or weeks in the past. See 
Leuchtkafer Letter 4, supra note 66, at 6.
---------------------------------------------------------------------------

    The commenter claimed that Dynamic M-ELO would operate outside of 
established non-discretionary methods, which require ``fully disclosed 
procedures operating in a strictly linear, invariant, and deterministic 
fashion.'' \83\ Additionally, the commenter stated that Nasdaq would be 
exercising discretion with Dynamic M-ELO to alter a participant's 
material order terms.\84\ The commenter claimed that Nasdaq would be 
using undisclosed data such as the

[[Page 62860]]

buyer and seller counts and recent trade sizes.\85\ Further, the 
commenter stated:
---------------------------------------------------------------------------

    \83\ See Leuchtkafer Letter 2, supra note 58, at 5-6. See also 
Leuchtkafer Letter 3, supra note 60, at 5-7. See also Leuchtkafer 
Letter 4, supra note 66, at 6-9 ('' . . . . Nasdaq's rulebook won't 
set out the `totality of the discretionary feature' (I believe it 
can't, because the totality changes week-to-week and even minute-to-
minute) and it won't define the `hardcoded conditionality' of its 
feature (again, I believe it can't), and a market participant won't 
be able to `recreate on its own' what Dynamic M-ELO has done 
(participants can't--it's not even clear anyone will be able to, as 
discussed below) . . . . Dynamic M-ELO departs from decades of this 
progress. Its behavior will not be deterministic or invariant over 
time, and purposefully so.'').
    \84\ See Leuchtkafer Letter 2, supra note 58, at 8. See also 
Leuchtkafer Letter 3, supra note 60, at 5-7.
    \85\ See Leuchtkafer Letter 2, supra note 58, at 8. See also 
Leuchtkafer Letter 3, supra note 60, at 5-7; Leuchtkafer Letter 4, 
supra note 66, at 5-7.

that (a) by determining the universe of data the system consumes, 
(b) by programming how the system thinks, (c) by controlling and 
supplying the information with which it thinks, and (d) by setting 
the goals and programming the nature and extent of its actions, and 
when it does all this to determine (e) when and in which prescribed 
intervals to set an ever variable time-in-force term for an order, a 
term which (f) dictates when to expose an order to the market to 
find contra-side interest, then without question Nasdaq is 
exercising control, judgment, and discretion over its customer 
---------------------------------------------------------------------------
orders.

    In Amendment No. 1, Nasdaq added language to address these 
concerns.\86\ Among other things, Nasdaq stated that to the extent that 
the design of Dynamic M-ELO permits variation in the Holding Periods 
for such orders, it does so by design, and the ``mere fact that the 
System may apply different weights over time to the factors it uses to 
determine whether and by how much to vary a Holding Period does not 
mean that the System will act with discretion in the same sense that a 
human being could be said to be exercise independent judgment when 
deciding whether and how to handle an order.'' \87\ Additionally, 
Nasdaq stated the following in its second response to comments:
---------------------------------------------------------------------------

    \86\ See supra notes 42-46.
    \87\ See supra note 45 and accompanying text.

    It is also worth noting that presently, exchanges like Nasdaq 
already employ non-linear, non-deterministic functionalities, like 
the randomized timers it uses to resolve certain unavoidable race 
conditions that arise in the order handling process. Nasdaq employs 
these functionalities with the knowledge of the SEC, and without any 
suggestion that they somehow transform Nasdaq into a broker.\88\
---------------------------------------------------------------------------

    \88\ See Nasdaq Second Response to Comments, supra note 61, at 
5-9. See also Nasdaq First Response to Comments, supra note 57, at 
5-7.

Furthermore, as noted above, the Exchange represents that outside of 
set retraining periods, ``the System will operate pursuant to a 
mathematical algorithm from which it cannot deviate--an algorithm that 
is programmed to achieve pre-defined and pre-disclosed objectives.'' 
\89\ The Exchange explains that outside of the set retraining periods 
``the System will behave predictably from day to day, such that its 
decisions when presented with given set of facts and circumstances in a 
given security on day 1 should be the same as they would be on day 2.'' 
\90\
---------------------------------------------------------------------------

    \89\ See supra note 46 and accompanying text.
    \90\ See supra note 29.
---------------------------------------------------------------------------

    Based on Nasdaq's representations described above, Dynamic M-ELO 
would operate pursuant to pre-determined, programmed procedures that 
would dictate order interaction and the terms for trading for each 
Dynamic M-ELO order entered on the Nasdaq trading facility. While the 
Exchange's procedures include conditions that, if satisfied under 
certain circumstances, might result in different outcomes for different 
M-ELO orders, such conditions and circumstances, if pre-determined, 
pre-defined, and programmed into the Exchange's trading facility, would 
be considered established and not discretionary. For example, according 
to the Exchange, Dynamic M-ELO may apply different pre-determined 
weights over time to pre-determined factors it uses to determine 
whether and by how much to vary a Holding Period.\91\ In such an event, 
Dynamic M-ELO will operate pursuant to pre-determined procedures and 
programmed mathematical algorithm from which it cannot deviate to 
``achieve pre-defined and pre-disclosed objectives.'' \92\ Further, the 
procedures governing Dynamic M-ELO and use of M-ELO orders will be 
established before the beginning of each trading day. For example, 
Dynamic M-ELO will use preset methods to evaluate and weigh specific 
data elements to determine the dynamic holding periods. Such pre-set 
methods will be established during the prior retraining period, and 
outside regular trading hours, and will not vary intra-day until 
adjusted at the next retraining period.
---------------------------------------------------------------------------

    \91\ See supra note 45 and accompanying text.
    \92\ See supra note 46 and accompanying text.
---------------------------------------------------------------------------

    Given the pre-determined, programmed procedures and rules that 
Nasdaq has proposed to dictate trading for Dynamic M-ELO, the 
Commission does not believe that Dynamic M-ELO is designed to provide 
Nasdaq with judgement and flexibility, and therefore, discretion over 
the handling or execution of a M-ELO order entered on the Exchange.

Unfair Discrimination

    Section 6(b)(5) of the Act requires that the rules of a national 
securities exchange not be designed to permit unfair discrimination 
between customers, issuers, brokers, or dealers. In several letters, 
one commenter stated that Nasdaq inadequately explains how it will 
monitor and, if necessary, adjust Dynamic M-ELO to ensure no unfair 
discrimination.\93\ Initially, this commenter emphasized what they 
perceived to be silence on the part of Nasdaq with regard to whether 
Dynamic M-ELO will discriminate among categories of participant 
types.\94\ In its response to these comments, Nasdaq initially added a 
new representation to the filing in Amendment No. 1, stating that that 
Dynamic M-ELO is not designed to further the performance of any 
participant or any category of participant, but instead has twin 
objectives--the absolute values of mark-outs and fill rates. In 
Amendment No. 2, Nasdaq expanded on this representation by adding the 
following:
---------------------------------------------------------------------------

    \93\ See Leuchtkafer Letter 3, supra note 60, at 7-8; 
Leuchtkafer Letter 2, supra note 58, at 5; Leuchtkafer Letter 1, 
supra note 48, at 3. Nasdaq's White Paper includes a ``firm-level 
analysis'' that ``tried to identify patterns and trends that could 
potentially signify a systematic bias towards specific firms.'' 
White Paper at 24. This analysis concluded that ``Dynamic M-ELO will 
not result in systematic-biased execution towards any one firm.'' 
Id. at 26.
    \94\ See Leuchtkafer Letter 3, supra note 60, at 7-8.

    Furthermore, Nasdaq performed internal tests of its AI model to 
detect indications of harmful bias in its performance results, and 
such tests concluded that no such indications exist. That is, the 
Exchange reviewed the impact on fill rates and mark-outs of Dynamic 
M-ELO, as compared to the ``static'' M-ELO, for those firms that 
accounted for more than 95% of M-ELO activity on the Exchange during 
Q1 2022 . . . . The Exchange analyzed results both in an absolute 
and a relative sense. Testing revealed that all participants 
experienced at least some improvements in fill rates and mark-outs 
when using Dynamic M-ELO versus static M-ELO, with the volume-
weighted average improvement being aligned with the results 
expressed in the White Paper. We detected no material variations 
that might suggest that a particular participant or category of 
participant (i.e., nature of firm; size of firm) benefitted from 
Dynamic M-ELO functionality to an extent that was unreasonably 
disproportionate to the benefits that other participants 
experienced. Thus, Nasdaq believes the model is objective, is 
---------------------------------------------------------------------------
designed to, and does avoid bias and discrimination.

    In Amendment No. 2, Nasdaq also affirmed that it will periodically 
review its model to ensure that it continues to perform in accordance 
with the Exchange's rules and that it has not introduced any harmful 
bias in favor of or against any participant or class of 
participants.\95\
---------------------------------------------------------------------------

    \95\ See Amendment No. 2 at fn. 34.
---------------------------------------------------------------------------

    In response to the above, the commenter submitted a fourth comment 
letter, in which they questioned the approach Nasdaq took to 
demonstrate that there is not bias against any one

[[Page 62861]]

participant or class of participants.\96\ The commenter, among other 
things, expressed concern about Nasdaq conducting its analysis using 
data for firms that accounted for 95% of M-ELO activity during Q1 of 
2022 rather than all M-ELO activity. The commenter states that Nasdaq 
did not describe how it determined the 5% of activity during that 
period to exclude from its analysis.\97\ For example, the commenter 
states that it is not clear whether Nasdaq excluded firms with large 
orders and trades, and the commenter opines that discarding any data 
could exclude activity that has qualitative or quantitative differences 
from the rest.\98\
---------------------------------------------------------------------------

    \96\ See Leuchtkafer Letter 4, supra note 66, at 1-4.
    \97\ See id. at 2-3 (``It seems Nasdaq trimmed its data before 
analyzing it for bias and constrained its analysis to `those firms 
that accounted for more than 95% of M-ELO activity on the Exchange 
during Q1 2022.' (I assume Nasdaq used the same data defined in the 
Filing as the `Training Period' for its analysis. Nasdaq doesn't say 
so, however.) Nasdaq doesn't describe the kind of M-ELO `activity' 
it filtered the data for, and specifically whether it filtered on 
order or trade counts or order or trade volume or some combination 
of two or more of these categories, or on some other factor, before 
removing firms from its analysis.'')
    \98\ See id. at 2-3.
---------------------------------------------------------------------------

    In response to this comment, Nasdaq represented that it conducted a 
supplemental analysis of the initially-excluded data--which were the 
activity of the least-active M-ELO firms from the control period of its 
initial analysis--to confirm whether its initial conclusions held for 
those participants.\99\ Nasdaq explains that the individual variations 
among the previously excluded participants was higher than that for the 
original batch of data, but that, based on simulated data, each of 
these participants would have experienced the same or better fill rates 
during the testing period if they had utilized Dynamic M-ELO.\100\ 
Based on this supplemental data analysis, Nasdaq concluded that there 
is no apparent biases for the Dynamic M-ELO, even among the least 
active M-ELO participants.\101\
---------------------------------------------------------------------------

    \99\ See id. at 5-6.
    \100\ See id.
    \101\ See id.
---------------------------------------------------------------------------

    The Commission concludes that Nasdaq has adequately demonstrated 
that the proposal is not designed to permit unfair discrimination 
consistent with Section 6(b)(5) of the Act. Through the White Paper, 
amendments, and response letters, Nasdaq has demonstrated that it has 
analyzed the anticipated or simulated effects of the proposed change on 
all current M-ELO users, and that this work did not indicate that 
particular firms or classes of firms are anticipated to unfairly 
benefit from or be harmed by the proposed Dynamic M-ELO functionality.

Prevention of Fraudulent and Manipulative Acts and Practices, Just and 
Equitable Principles of Trade, and the Protection of Investors and the 
Public Interest

    Section 6(b)(5) of the Act also requires that the rules of a 
national securities exchange be designed to prevent fraudulent and 
manipulative acts and practices, to promote just and equitable 
principles of trade, and, in general, to protect investors and the 
public interest. In Amendment No. 2, Nasdaq addressed whether the 
Dynamic M-ELO is designed to prevent fraudulent and manipulative acts 
and practices. Nasdaq states that the design of Dynamic M-ELO does not 
lend itself to potential manipulation by a single participant or a 
small group of participants because the machine-learning model makes 
determinations regarding Holding Periods based upon prevailing market-
wide conditions for a given symbol, rather than the behaviors of 
particular participants with respect to that symbol, or the activity of 
participants in M-ELOs involving that symbol. Nasdaq further states 
that manipulation of the machine-learning model would be difficult to 
accomplish given the large number of variables that factor into the 
machine-learning model's decisions to change Holding Periods during 
Change Events, as well as the different weights that apply to each such 
factor, which as described above, may vary over time. Furthermore, 
Nasdaq states that any benefits that a participant might derive from 
manipulating the duration of Holding Periods would likely be small and 
outweighed significantly by the difficulty and cost of effecting such 
manipulation.
    The Exchange, in Amendment No. 2, also sets forth representations 
regarding how it will surveil its market after Dynamic M-ELO is 
implemented. First, Nasdaq represents that it will review the machine-
learning functionality and operation periodically to affirm that it 
continues to perform in accordance with the Exchange's rules and has 
not introduced any harmful bias in favor of or against any participant 
or category of participants.\102\ Nasdaq also represents above that it 
will surveil for indications of manipulation and act accordingly if it 
detects such indications.
---------------------------------------------------------------------------

    \102\ See supra note 41.
---------------------------------------------------------------------------

    The Commission finds that the proposed rule change is designed to 
prevent fraudulent and manipulative acts and practices consistent with 
Section 6(b)(5) of the Act. The Commission agrees that aspects of the 
Dynamic M-ELO design reduce opportunities for manipulation or are 
likely to make manipulation costly or difficult. The model's operation 
depends on 142 data elements, which are each likely to have effects on 
model outcomes of differing magnitudes and in different directions. 
Many of these data elements are also based on market-wide data, in some 
cases spanning periods of days,\103\ which are likely themselves 
difficult for market participants to manipulate. Given these design 
features, it appears likely that manipulating the duration of Dynamic 
M-ELO holding periods in any given symbol or group of symbols would be 
an extremely complex undertaking. In light of this complexity, and the 
size of M-ELO activity relative to the market for NMS stocks,\104\ 
Nasdaq's assertion that the potential benefits of manipulating the 
dynamic holding periods for these order types would be outweighed by 
the cost and complexity of manipulation also appears reasonable. Nasdaq 
has also represented that it intends to surveil the proposed order 
types for manipulation. This ongoing surveillance, to ensure the 
appropriate use of Dynamic M-ELO by Exchange Members and behavior by 
the machine-learning model, is important to the successful 
implementation of Dynamic M-ELO and appears appropriately tailored to 
the accomplish the intent of the M-ELO and M-ELO+CB order types.
---------------------------------------------------------------------------

    \103\ See, e.g., Exhibit 3b.
    \104\ According to the ``M-ELO Monthly Report'' published by 
Nasdaq for July 2023 (available at: https://www.nasdaq.com/docs/M-ELO-Monthly-Report (accessed September 2, 2023)), the average daily 
notional volume executed in M-ELO was $624,556,748. The average 
daily notional volume executed in July 2023 across the market for 
NMS stocks was about $523,769,246,196. See, e.g., Cboe, Historical 
Market Volume Data, available at: https://www.cboe.com/us/equities/market_statistics/historical_market_volume/. The average daily 
notional volume in M-ELO for that month was approximately 0.12% 
(just over one-tenth of one percent) of the average daily notional 
volume across the entire NMS stock market.
---------------------------------------------------------------------------

    Furthermore, the Commission finds that overall structure of Dynamic 
M-ELO--particularly, the static numerical constraints set forth in the 
proposed rule text--is designed in general, to protect investors and 
the public interest and promote just and equitable principles of trade 
pursuant to Section 6(b)(5) of the Act. As described above, the model 
will continuously engage in dynamic analysis of current market 
conditions during trading hours, and outside of market hours, it will 
retrain with the goal of improving the overall performance of Dynamic 
M-ELO. These dynamic aspects of the proposal,

[[Page 62862]]

however, are constrained by the static numerical thresholds set forth 
in the proposed rule text. For example, the initial Holding Periods for 
each trading day will be 1.25 milliseconds, the overall range for any 
Holding Period must be between 0.25 and 2.50 milliseconds during normal 
market conditions, and the Holding Period can only change by either 
0.25 or 0.50 milliseconds at each Change Event during normal market 
conditions. Regardless of how the model analyzes the current market or 
changes the weighting of the data elements as a result of its 
retraining, Dynamic M-ELO cannot operate outside of the static 
numerical ranges and limitations or minimums set forth in the rule 
text. As such, the Commission finds that Nasdaq has designed Dynamic M-
ELO to operate in a manner that in general protects investors and the 
public interest and promotes just and equitable principles of trade in 
accordance with Section 6(b)(5) of the Act.

Compliance With SRO Recordkeeping and Reporting Obligations

    One commenter queried whether Nasdaq could maintain an adequate 
audit trail given the potential for frequently shifting Holding Periods 
for Dynamic M-ELO.\105\ In response, Nasdaq states that it will retain 
copies of each iteration of its system as part of its books and records 
and will disclose publicly statistics relating to Dynamic M-ELO 
performance.\106\ Nasdaq additionally represented that it will publish 
weekly and monthly Dynamic M-ELO performance statistics, which would 
include the weekly numbers of shares and trades in M-ELOs by symbol, 
weekly aggregated M-ELO share and trade data, and monthly aggregated 
block data, on Nasdaqtrader.com.\107\ Nasdaq also indicated it would 
add statistics to its existing M-ELO Monthly Report, which discloses 
quote stability by time horizon, about how frequently, on average, its 
system changes Holding Period durations for the top decile, median, and 
bottom decile of symbols, as measured by monthly M-ELO and M-ELO+CB 
trading volumes.\108\
---------------------------------------------------------------------------

    \105\ See Leuchtkafer Letter 1, supra note 48, at 1.
    \106\ See Nasdaq First Response to Comments, supra note 57, at 
3.
    \107\ See Section III.A.1., supra.
    \108\ See id.
---------------------------------------------------------------------------

    Nasdaq also added a representation to the filing, addressing how it 
would comply with its recordkeeping obligations.\109\ Nasdaq states 
that it will retain copies of each historical iteration of its models 
as part of its books and records, and make them available to the 
Commission upon request, should it wish to examine them to understand 
how the model changes over time.\110\ Nasdaq also states that it will 
publish an equity trader alert in advance of deploying a retrained 
version of Dynamic M-ELO when Nasdaq anticipates the retrained version 
will produce results that differ materially from the prior 
version.\111\ Based on these representations, the Commission finds that 
Nasdaq has met its burden to demonstrate that it will comply with all 
relevant exchange recordkeeping requirements and obligations when it 
implements Dynamic M-ELO. In addition, the Commission notes that Nasdaq 
must comply with its reporting obligations under Rule 613 of Regulation 
NMS \112\ and the National Market System Plan Governing the 
Consolidated Audited Trail (``CAT NMS Plan'') \113\ with respect to 
Dynamic M-ELO, which requires it to record and electronically report to 
the central repository the material terms of each order and each 
reportable event.\114\
---------------------------------------------------------------------------

    \109\ See id.
    \110\ See id.
    \111\ See id.
    \112\ See 17 CFR 242.613.
    \113\ The CAT NMS Plan was approved by the Commission, as 
modified, on November 15, 2016. See Securities Exchange Act Release 
No. 79318 (November 15, 2016), 81 FR 84696 (November 23, 2016).
    \114\ See 17 CFR 242.613(c)(7).
---------------------------------------------------------------------------

Nasdaq's Obligation To File Proposed Rule Changes Relating to Dynamic 
M-ELO

    Prior to the filing of Amendment No. 1, a commenter stated that it 
was unclear what types of changes to the model would lead Nasdaq to 
seek approval from the Commission via an SRO rule filing.\115\ As 
explained above,\116\ Nasdaq represents that it will not modify the 
underlying structure of Dynamic M-ELO without first obtaining the 
Commission's approval to do so, including modifications to the data 
elements the model considers in making decisions about Holding Period 
durations, the conditions under which the model may adjust the duration 
of Holding Periods, the frequency with which the model may adjust the 
Holding Periods, the range of Holding Period durations available to M-
ELOs and M-ELO+CBs, the increments by which Holding Periods may change 
at any given Change Event, and the procedures for triggering, 
maintaining, and ending 12 millisecond Holding Periods during times of 
extraordinary instability. In contrast, the Exchange states that it 
will not seek Commission approval prior to retraining the model to 
adjust the weighting it applies to those data elements pursuant to the 
weekly retraining process.
---------------------------------------------------------------------------

    \115\ See Leuchtkafer Letter 1, supra note 48, at 2.
    \116\ See Section II.A.1., supra.
---------------------------------------------------------------------------

    Section 19(b)(1) of the Act \117\ and Rule 19b-4 thereunder \118\ 
require an SRO to file a proposed rule change with the Commission 
whenever it seeks any proposed change in, addition to, or deletion from 
the rules governing the SRO and its members' activities on the SRO. As 
discussed above, the proposal sets forth the specific data elements 
that Dynamic M-ELO will use during the trading day. Furthermore, the 
proposed rule change sets forth when the machine-learning model will 
retrain and the extent to which the retraining can and cannot cause the 
machine-learning model to update Dynamic M-ELO's operation during 
subsequent trading days.\119\ In addition, the proposal sets forth the 
operation of Dynamic M-ELO, such as the potential range for a Holding 
Period, how often Dynamic M-ELO reevaluates market conditions for a 
given security to adjust a Holding Period, and the increment by which a 
Holding Period may be changed. Nasdaq represents that it will not 
change any of these aspects of the proposal or any other function of 
Dynamic M-ELO without first filing a proposed rule change.\120\ Nasdaq 
does, however, state that it would not file a proposed rule change in 
connection with the operation of the machine-learning model's weekly 
retraining and the results of that process.
---------------------------------------------------------------------------

    \117\ 15 U.S.C. 78s(b)(1).
    \118\ 17 CFR 240.19b-4.
    \119\ See Section III.A.1., supra.
    \120\ See supra note 30 and accompanying text.
---------------------------------------------------------------------------

    Based on the foregoing, the Commission believes that Nasdaq has 
adequately responded to the commenter's concern. Nasdaq will need to 
file a proposed rule to make any changes, additions, or deletions to 
the operation of Dynamic M-ELO as approved herein. Nasdaq has 
delineated when it would file a proposed rule change to alter the 
operation of Dynamic M-ELO, and when the machine-learning model would 
retrain and adjust the weighting it applies to the data elements 
without it filing a proposed rule change. Specifically, Nasdaq's 
proposed rule change and rule text reflect the 142 data elements 
Dynamic M-ELO will consider when determining the Holding Period for a 
security and the goals Nasdaq will consider when weighing those data 
elements (i.e., reducing mark-outs and increasing fill rates) but does 
not set forth the relative weighting of each those individual data 
elements. Though

[[Page 62863]]

the structure of the proposal does not disclose of the exact weighting 
for each of the 142 data elements, it does set forth the two goals 
Nasdaq will consider when weighing those data elements initially and 
during each weekly retraining, which provides information as to how 
those 142 factors will be used in determining the Holding Period for a 
security. Based on how the proposed rule sets forth the goals that will 
govern each retraining, the Commission believes that Nasdaq's 
delineation of when it would and would not file a proposed rule change 
to alter the operation of Dynamic M-ELO is consistent with Nasdaq's 
rule filing obligation. The Commission agrees that the weekly 
retraining to optimize the weighting of the 142 data elements 
considered by Dynamic M-ELO to best achieve those goals within the 
rule's parameters would not necessitate the filing of a proposed rule 
change with the Commission because those adjustments would be 
reasonably and fairly implied by the proposed rule. However, to the 
extent Nasdaq seeks to change, add to, or delete from the rule's 
construct in connection with the weekly retraining, it would first be 
required to file a proposed rule change with the Commission.

V. Solicitation of Comments on Amendment No. 2 to the Proposed Rule 
Change

    Interested persons are invited to submit written data, views, and 
arguments concerning whether Amendment No. 2 is consistent with the 
Act. Comments may be submitted by any of the following methods:

Electronic Comments

     Use the Commission's internet comment form (https://www.sec.gov/rules/sro.shtml); or
     Send an email to [email protected]. Please include 
file number SR-NASDAQ-2022-079 on the subject line.

Paper Comments

     Send paper comments in triplicate to Secretary, Securities 
and Exchange Commission, 100 F Street NE, Washington, DC 20549-1090.

All submissions should refer to file number SR-NASDAQ-2022-079. This 
file number should be included on the subject line if email is used. To 
help the Commission process and review your comments more efficiently, 
please use only one method. The Commission will post all comments on 
the Commission's internet website (https://www.sec.gov/rules/sro.shtml). Copies of the submission, all subsequent amendments, all 
written statements with respect to the proposed rule change that are 
filed with the Commission, and all written communications relating to 
the proposed rule change between the Commission and any person, other 
than those that may be withheld from the public in accordance with the 
provisions of 5 U.S.C. 552, will be available for website viewing and 
printing in the Commission's Public Reference Room, 100 F Street NE, 
Washington, DC 20549 on official business days between the hours of 
10:00 a.m. and 3:00 p.m. Copies of the filing also will be available 
for inspection and copying at the principal office of the Exchange. Do 
not include personal identifiable information in submissions; you 
should submit only information that you wish to make available 
publicly. We may redact in part or withhold entirely from publication 
submitted material that is obscene or subject to copyright protection. 
All submissions should refer to File Number SR-NASDAQ-2022-079, and 
should be submitted on or before October 4, 2023.

VI. Accelerated Approval of Proposed Rule Change, as Modified by 
Amendment No. 2

    The Commission finds good cause to approve the proposed rule change 
prior to the 30th day after the date of publication of Amendment No. 2 
in the Federal Register. Amendment No. 2 does not include any material 
changes to the operation of the proposed Dynamic M-ELO and its machine-
learning model. In Amendment No. 2, the Exchange: (1) adds the defined 
term ``proprietary assessment of market conditions'' to the proposed 
rule text, which consolidates certain details and explanations about 
how the machine-learning model would operate from prior versions into a 
single defined term; (2) revises the list of factors provided in 
Exhibit 3b to include expanded and ``simplified'' explanations of the 
terminology used therein; (3) adds a representation that the systems 
used to operate Dynamic M-ELO and machine-learning model are ``SCI 
Systems'' and thus subject to compliance with Regulation SCI; and (4) 
expands the legal analysis to address comments regarding unfair 
discrimination and the exercise of impermissible discretion by the 
Exchange.
    The Commission finds that Amendment No. 2 raises no novel 
regulatory issues that have not previously been subject to comment and 
is reasonably designed to prevent fraudulent and manipulative acts and 
practices, to promote just and equitable principles of trade, and, in 
general, to protect investors and the public interest, and not be 
unfairly discriminatory, or impose an unnecessary or inappropriate 
burden on competition. Amendment No. 2 does not alter the proposed 
operation or any material features of Dynamic M-ELO, which operation 
and features have been subject to two rounds of public comment. In 
response to public comment, the revisions to the proposal contained 
within Amendment No. 2 provide additional clarification and details 
regarding how Dynamic M-ELO and the machine-learning model will 
operate, as well as additional legal analysis to support the Exchange's 
position that the proposal is consistent with the Act. Accordingly, 
pursuant to Section 19(b)(2) of the Act,\121\ the Commission finds good 
cause to approve the proposed rule change on an accelerated basis prior 
to the 30th day after publication of notice of the filing of Amendment 
No. 2 in the Federal Register.
---------------------------------------------------------------------------

    \121\ 15 U.S.C. 78s(b)(2).
---------------------------------------------------------------------------

VII. Conclusion

    It is therefore ordered, pursuant to Section 19(b)(2) of the 
Act,\122\ that the proposed rule change (SR-NASDAQ-2022-079), as 
modified by Amendment No. 2, be, and it hereby is, approved on an 
accelerated basis.
---------------------------------------------------------------------------

    \122\ 15 U.S.C. 78s(b)(2).

    For the Commission, by the Division of Trading and Markets, 
pursuant to delegated authority.\123\
---------------------------------------------------------------------------

    \123\ 17 CFR 200.30-3(a)(12).
---------------------------------------------------------------------------

Sherry R. Haywood,
Assistant Secretary.
[FR Doc. 2023-19728 Filed 9-12-23; 8:45 am]
BILLING CODE 8011-01-P

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.