This is the information page for the 1 hour Quantum Computing Benchmarking Working Group (QCB-WG) P7131 meeting. The meeting is Tuesday, June 28, 2022 10 AM U. S. Eastern Daylight time via Zoom. The Zoom room will operate for a total of two hours if there is a need for informal discussion.
Main topic is talk:
A perspective from QED-C on Quantum Computing Standards and Benchmarks by Tom Lubinski, QED-C Standards Technical Advisory Committee (TAC)
The talk would be about what the QED-C Standards TAC has learned about the landscape for Quantum Computing standards and benchmarks, the various projects that have been initiated, completed and underway. This would be followed by its view to the future and ways to minimize fragmentation of similar efforts in the community.
Reference: Lubinski, Thomas, et al. “Application-oriented performance benchmarks for quantum computing.” arXiv preprint arXiv:2110.03137 (2021).
The plan is to have a P7131 business meeting in the first 15 minutes, including determination of quorum, etc. People not part of P7131 can join 15 minutes into the meeting when Tom Lubinski will start a presentation on the QED-C benchmark suite. The formal presentation is expected to be 15-20 minutes followed by discussion. The meeting will officially end after one hour, but discussion can continue via Zoom until the 2 hour point.
This meeting is open to all that are interested. All you need to attend is the Zoom connection information that appears at the bottom of this page. We will take attendance in accordance with IEEE-SA’s policies.
Agenda:
10:00 AM EDT: Call to order
10:00 AM EDT: Establish voting membership
10:02 AM EDT: Call for (a) approval of minutes from two past meetings (see below) and (b) changes to this this draft agenda, followed by approval.
10:05 AM EDT: IEEE-SA policies
10:15 AM EDT: Presentation on QED-C benchmark suite by Tom Lubinski.
10:35 AM EDT: Question and answer / discussion
11:00 AM EDT: Meeting officially adjourns
11:00 AM EDT: Discussion may continue if needed
noon EDT: Must end by this time
This webpage https://sagroups.ieee.org/7131/qcb-wg-qgl-28jun22/ is expected to be maintained, so check back
—————————–
Erik DeBenedictis is inviting you to a scheduled Zoom meeting.
Topic: QCB-WG P7131 Jun 28, 22 (topic: QED-C benchmark suite)
Time: Jun 28, 2022 08:00 AM Mountain Time (US and Canada)
Join Zoom Meeting
https://us02web.zoom.us/j/87971264493?pwd=ZXpUdWloZWpBK2sxV2Frb1JEZ0trZz09
Meeting ID: 879 7126 4493
Passcode: 644020
One tap mobile
+12532158782,,87971264493#,,,,644020# US (Tacoma) +13462487799,,87971264493#,,,,644020# US (Houston)
Dial by your location
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 9128 US (San Jose)
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
Meeting ID: 879 7126 4493
Passcode: 644020
Find your local number: https://us02web.zoom.us/u/kmg6LfvI9
——————————
Minutes February 28, 2022 meeting
(a) P7131/QCB-WG
10:00 AM EST (b) Meeting via Zoom called to order 10 AM EST (c) Chair Erik DeBenedictis, presiding and writing (these) minutes as secretary.
10:05 AM EST Introductions (d) and establishment of voting membership. The following are derived from contemporaneous notes and may contain spelling errors. The contemporaneous notes contain more data. The information in the chair’s contemporaneous notes has been merged with Jonathan Goldberg’s notes to create the QCB-WG and QCB-WG-MBRS LISTSERV group lists.
Mike Pendergrast. Vote.
Alexandru Paler. Vote.
Jonathan Goldberg. IEEE SA Staff.
Santanu Basu. Corning. Vote.
Bruce Kraemer. Vote.
Dick Barr. Vote.
John Spargo. Vote.
Ellen Winsor. Vote.
Alvin Chin.
Andrew Ochoa. No vote.
Baw Chng. Bawman LLC, reps self. Vote.
Brice Achir, Cisco. Voting.
Charlie Baldwin, Quantumuum. No vote.
Claudio Lima, BC Vote
Cathy Mcgeoch, D-Wave, vote.
Dan Strano. Unitary fund. Vote.
Dan Gottesman, Keysight/representing self.
Dipankar. Vote.
Esam El-Araby. Vote.
Florian Price. No vote.
Ivan Rungger. UK National Measurement. Vote.
Jennifer Harding. Q. Brilliance. Voting.
John Devaney. UK National phy laboratory. Vote.
John Greaves. Vote yes.
K.-C. Chen. Univ Southbury. Vote.
Kevin Young. Sandia. No vote.
Masahiro Horbe. Vote.
Maurizio Ferrari Dacrema. Post doc Polytech Milan. No vote.
Mingchange Wang. Hardware from Dell tech. Vote.
Phillip Ross. No vote.
Raj Chanian. UK cap ni? Vote.
Reena Dayal. IEEE QI also represent Q. Ecosystems Tech council of India. No vote.
Rajesh. Mo… Management consulting company. No vote.
Sara Biyabani. From Si. Valley? Vote.
Scott Glancy. NIST. Vote yes.
Scott Holmes. Works for BAH. Vote.
SD. Das. India. No vote.
Sebastian. Rockwell. No vote.
Seth Merkel. Physicist IBM. Vote yes.
Simon Caton. Asst Prof CS U. College Dubin. Vote.
Sri Chan… IEEE staff. No vote.
Steve Bush. GE Research. No vote.
Surat Hazra “gam genini”? No vote.
Terrill Frantz. Harrisburg U. Vote.
Victor Rodriguez-Toro. IBM.
Victoria. Certified auditor. Vote.
Vladimir Getov. CS Prof U. Westminister London. Vote.
Yan-Ping Lu. Monash Univ. Vote.
Angie Quarry. Vote.
Paolo Gargini. Yes vote.
Sara Kaiser. Unitary fund. Voting.
Fujitsu Hiroki, Itoh. No vote.
10:00 AM EST (e) N/A, this was a kickoff meeting.
11:05 AM EST Call for changes to agenda: none
11:05 AM EST(f) Approval of agenda: Motion McGeoch Second Winsor, pass unanimous consent
11:08 AM EST Jonathan Goldberg read (g) patent, (h) copyright, participation slide decks
(i) no previous meeting.
DeBenedictis, Vladimir Getov, Angie Qarry, Dick Barr, Cathy McGeoch
(l) There was no executive session.
11:10 AM EST. Erik DeBeneductis discussed slide deck (j)
No action items (k), no executive session (l)
Motion to end meeting: Motion Baw Second Barr pass unanimous consent
11:20 AM EDT End of informal discussion
(n) The next meeting will be scheduled by email, Doodle, etc. (but there was no discussion here).
Minutes May 10, 2022
(a) P7131/QCB-WG
9:00 AM EDT (b) Meeting via Zoom called to order 9 AM EDT (c) Chair Erik DeBenedictis, presiding and writing (these) minutes as secretary.
9:00 AM EDT (d) Establish voting membership (e) It was initially assessed there were not enough members present, but a more careful tally after then meeting revealed that late arrivals resulted in the exact minimum quorum (see attendance list). No votes were needed.
Attendence list:
Abhishek Agarwal, National Physical Laboratory, UK
Jonathan J. Attia, Feynman Foundation
Richard (Dick) Barr, Southern Methodist University
Simon Caton, University College Dublin
Raj Chanian, Capgemini
Baw Chng, BAWMAN LLC
Shamik Das, The MITRE Corporation
Shouvik Datta, IISER-Pune
Erik DeBenedictis, self
John Devaney, National Physical Laboratory UK
Esam El-Araby, University of Kansas (KU)
Terrill Frantz, Harrisburg University
Joshua Gay, IEEE
Vladimir Getov, University of Westminster
Victoria (Vicky) Hailey, vhg
Jennifer Harding, Quantum Brilliance
Hiroshi Kawamura, Fujitsu ltd.
Bruce Kraemer
Catherine McGeoch, dwavesys.com
Alexandru Paler
Angie Qarry, Disruptive Innovations
John Spargo, Northrop Grumman
Wen Wei, Fujitsu Limited
9:05 AM EDT Call for changes to agenda: none
9:05 AM EDT(f) Approval of agenda: Motion McGeoch Second Barr pass unanimous consent
9:08 AM EDT Read (g) patent, (h) copyright, participation slide decks
(i) Oops, missed approval of minutes of previous meeting.
9:15 AM EDT (j) Introduce Silona Bonewald, who presented the slide deck below:
“Assignments” (k):
Cathy to send template for benchmark to Erik; Erik to forward to QCB-WG; WG email reflector to be used to assemble first cut at a list of benchmarks
Initial group for assembling a list of benchmarks and possible some speakers at meeting: Erik DeBenedictis, Vladimir Getov, Angie Qarry, Dick Barr, Cathy McGeoch
(l) There was no executive session.
9:59 AM EDT (m) Motion to end meeting: Motion Barr Second McGeoch pass unanimous consent
10:20 AM EDT End of informal discussion
(n) The next meeting will be scheduled by email, Doodle, etc. (but there was no discussion here).
Minutes June 28, 2022
(a) P7131/QCB-WG
10:00 AM EDT (b) Meeting via Zoom called to order 10 AM EDT (c) Chair Erik DeBenedictis, presiding and writing (these) minutes as secretary.
10:00 AM EDT (d) Establish voting membership (e) It was initially assessed there were not enough members present, but a more careful tally after then meeting revealed that 19 voting members attended, which met the minimum quorum (see attendance list). However, there were not enough members present at the beginning of the meeting to approve past minutes.
Attendance list:
Erik DeBenedictis
Bruce Kraemer
Shouvik Datta
Esam El-Araby
Jonathan Goldberg
Richard Barr
Simon Caton
Terrill Frantz
Vladimir Getov
Anne Sophie Barbe
Victoria (Vicky) Hailey
Angie Qarry
Ivan Rungger
Tom Lubinski
Jennifer Harding
brice achkir
santanu basu
Panu Vesanen
Masahiro Horibe
Catherine McGeoch
Claudio Lima
Raj Chanian
Baw Chng
Kazutomo Hasegawa
Skamik Das
Yan-Ping Lu
John Qu
8:05 AM EDT Call for changes to agenda: none
8:05 AM EDT(f) Approval of agenda: [forgot about this step]
9:08 AM EDT Read (g) patent, (h) copyright, participation slide decks
(i) No approval of minutes of previous meeting to do lack or quorum as of this time.
9:15 AM EDT (j) Introduce Tom Lubinski, who presented the slide deck below:
[[*** TOM’s SLIDE DECK ***]]
Tom: Discussion of IEEE 7131. Vetting QED-C approach
Erik: Discussion of trial-use standard when real standard premature
Esam says state preparation and measurement is missing from existing benchmarks and should be added. Tom Lubinski says this is a great idea.
Angie: IEEE and QED-C could get voice of industry involved in what benchmarks are needed. How to get use more involved.
Vicky: Trial use standards are tech reports for ISO; this group similar to ISO AI group; when IEEE fast tracks to ISO, ISO can change.
Vladimir: Liked presentation. Should call what’s going on now (execution time measurements) a “methodology.” Tom: QED-C focus has been comparing different platforms.
9:04: (m) Motion to adjourn Cathy, seconded Dick, passed unanimously (however, meeting continues informally for another 30 mins).
Angie: Corporations interested in runtime. For example quantum 1 minute, classical 3 minutes.
Tom on action items for QED-C & P7131: (1) QED-C has an open source repository for those who can run benchmarks (2) Joint meeting between IEEE + QED-C contributors
Simon: Quantum machine learning used data sets (will provide link to a paper) Conclusions: Shallow circuits favored for reasons beyond just low noise.
Cathy: Part of a group doing an “industry manifesto.” What is sought from “makers” is metrics in $ (money) not runtime.
Meeting ends.
“Assignments” (k): Discussion of a meeting with more people, P7131 access to QED-C repository, and IEEE could vet QED-C benchmarks.
(l) There was no executive session.
9:40 AM EDT End of informal discussion
(n) The next meeting will be scheduled by email, Doodle, etc. (but there was no discussion here).