IEEE 3333.1
Standard for the Quality Assessment of Three Dimensional (3D) Displays, 3D Contents and 3D Devices based on Human Factors

Policies & Procedures

P3333.1 – approved by C-SAB – 20200812


Policies and Procedures for: 3D Human Factors Working Group

Date of Approval:  August 12, 2020



3D Human Factors Working Group Policies and Procedures for Standards Development


  • Introduction


1.1 Role of Standards Development and these Procedures



In today’s technological environment, standards play a critical role in product development and market competitiveness. In the IEEE, the responsibility for how a standard originates and evolves is managed by a Sponsor. It is essential in the management of a standard’s development to avoid any actions by the Sponsor or the participants that result in a violation of procedures. These procedures establish the necessary framework for a sound standardization process.


1.2 Conduct


Meeting attendees and participants in standards activities shall demonstrate respect and courtesy toward each other and shall allow each participant a fair and equal opportunity to contribute to the meeting discussion. While participating in IEEE standards development activities, all participants, including but not limited to, individuals, entity representatives, entity members, entities participating directly in the entity process, and entities participating indirectly in the individual process shall act in accordance with all applicable laws (nation-based and international), the IEEE Code of Conduct, the IEEE Code of Ethics, and with IEEE-SA Standards Board Bylaws (see IEEE-SA Standards Board Bylaws Clause 5.2.1 on “Participation in IEEE standards development”) and IEEE-SA Standards Board Operations Manual.


Membership privileges can be lost through persistent violation of the fundamental principles of operation or disregard of standards of conduct.


A Working Group Chair that suspects persistent violation of these principles or standards by an individual shall refer the matter to the Sponsor.


1.3 Modifications to These Procedures

These Policies and Procedures outline the orderly transaction of business by the 3D Human Factors Working Group, hereinafter referred to as “the Working Group”.

The Working Group may amend these procedures with the approval of its Sponsor. The Sponsor may modify these procedures. Modification in this context means that material in these procedures may be modified as long as that clause is not indicated as one that shall not be changed. The IEEE-SA Audit Committee (AudCom) strongly recommends that all subjects included in these procedures are addressed by the Working Group or Sponsor.

None of the rules or requirements in these policies and procedures may be suspended.


1.4 Hierarchy


Participants engaged in the development of standards shall comply with applicable federal, state, and international laws. In addition, for standards matters, the latest version of several documents takes precedence over these procedures in the following order:


New York State Not-for-Profit Corporation Law

IEEE Certificate of Incorporation

IEEE Constitution and Bylaws

IEEE Policies

IEEE Board of Directors Resolutions

IEEE Standards Association (IEEE-SA) Operations Manual

IEEE-SA Board of Governors Resolutions

IEEE-SA Standards Board Bylaws

IEEE-SA Standards Board Operations Manual

IEEE-SA Standards Board Resolutions


Policies and Procedures for Standards Development for the IEEE Computer Society Standards Activities Board Standards Committee


Robert’s Rules of Order Newly Revised (RONR) is the recommended guide on questions of parliamentary procedure not addressed in these procedures.


1.5 Fundamental Principles of Operation


For the development of standards, openness and due process are mandatory.


Openness requires that any person who has, or could be reasonably expected to have an interest, and who meets the requirements of these procedures, has a right to participate by:


  1. Attending Working Group meetings (in person or electronically)
  2. Becoming a member of the Working Group
  3. Becoming an officer of the Working Group
  4. Expressing a position and its basis,
  5. Having that position considered, and
  6. Appealing if adversely affected.


IEEE due process requires a consensus of those parties interested in the project. Consensus is defined as at least a majority agreement, but not necessarily unanimity.


Due process is based upon equity and fair play. In addition, due process requires openness and balance (i.e., the standards development process shall strive to have a balance of interests and not to be dominated by any single interest category). However, for the IEEE-SA Sponsor ballot, there shall be a balance of interests without dominance by any single interest category.


1.6 Definitions


Written communication includes but is not limited to meeting minutes, letter, email, and fax.

Sponsors of IEEE standards projects are committees that are responsible for the development and coordination of the standards project and the maintenance of the standard after approval of the standard by the IEEE-SA Standards Board. (IEEE-SA Standards Board Bylaws 5.2.2).

Responsible Subcommittee is an optional subgroup of the Sponsor with delegated responsibility for approving PARs and Sponsor ballots. Responsible Subcommittees assist the Sponsor committee in the control and management of a large scope of work involving multiple working groups. A PAR study group or a working group (WG) is not a Responsible Subcommittee.

In this document, the term “Sponsor” means the sole Sponsor, the Responsible Subcommittee, or in the case of co-sponsored projects, the primary Sponsor.


A participant is an individual involved in the standards development process (see IEEE-SA Standards Board Bylaws Clause 5.2.1 on “Participation in IEEE standards development”). NOTE: Participants can be members or non-members of IEEE, IEEE-SA and/or of the working group.


A non-member is a participant who has not satisfied the criteria for membership defined in 4.1.


A member is a participant who has satisfied the criteria for membership defined in 4.1. A member is eligible to be listed on the WG roster.


A voting member is a member who has satisfied the criteria for voting membership defined in 4.1. A voting member can participate in working group motions and ballots.


A non-voting member is a member who has not satisfied the criteria for voting membership defined in 4.1.


The relationship between the terms participant, non-member, member, voting member and non-voting member is illustrated in the following diagram:





2.0 Working Group Responsibilities


The Working Group shall:


  1. Complete the project from Project Authorization Request (PAR) approval to IEEE-SA Standards Board approval as specified by the PAR, and in compliance with IEEE-SA policies and procedures.
  2. Use the IEEE-SA document template format [or the applicable template for standards jointly developed with, or adopted from, another international standards organization].
  3. Submit to the Sponsor any documentation required by the Sponsor; e.g., a project schedule or a monthly status report.
  4. Notify the Sponsor of the draft development milestones.
  5. Notify the Sponsor when the draft is ready to begin IEEE-SA Sponsor ballot.
  6. Only those authorized to access and use IEEE’s data, including personal data, from IEEE systems are permitted to do so, for the purposes intended, including to support the technical development work on the standard, and only in compliance with IEEE or IEEE-SA Privacy and data privacy policies.


3.0 Officers


There shall be a Chair and a Secretary, and there should be a Vice-Chair. The office of Treasurer is suggested if significant funds are involved in the operation of the Working Group and/or its subgroups or if the group has multiple financial reports to supply to the IEEE-SA. A person may simultaneously hold the positions of Treasurer and another office, other than Chair.


The officers (and any person designated to manage the Sponsor ballot) shall each be IEEE members of any grade, or IEEE Society affiliates, and also be members of IEEE-SA.


3.1 Election or Appointment of Officers


Officers shall be determined in accordance with the procedures of the sponsor. The procedures are as follows:


The Working Group Chair shall be appointed in accordance with the Sponsor’s procedures.


The Working Group Chair, or the sponsor, shall appoint an Elections Administrator whose function is to gather nominations for other officers, and conduct an election. The Elections Administrator shall not be a nominee in the election. The election for officers shall be conducted annually.


Working Group voting members shall nominate to the Elections Administrator one or more voting members for the positions of Vice-Chair, and Secretary.


Nominees shall be eligible to hold the office, if and when they are elected (see Clause 3.0). A person shall be nominated for no more than one office, except in the case of Treasurer, as per Clause 3.0. Upon written notification, the nominee shall, within 14 calendar days, indicate acceptance or rejection of the nomination. If no nomination is received or accepted for an office, the Working Group Chair may appoint a voting member for the office.


The Elections Administrator shall prepare and conduct the election by letter or electronic ballot. Voting will conclude in a time determined by the Working Group, but no less than 14 calendar days. Each voting member shall cast a single election ballot. Within the ballot, the voting member may vote for multiple nominees for each office with the limit of one vote per nominee. The nominee with the greatest number of approval votes shall win the election, provided ballots are returned by a majority of the eligible voters for that election.


The term of office for each officer shall be one year, but an officer may serve until a successor is selected.


3.2 Temporary Appointments to Vacancies


If an office other than the Chair is vacant for any reason (such as resignation, removal, lack of nomination at an election), a temporary appointment shall be made by the Chair for a period of up to 12 months. In the case of Chair, the Sponsor shall make the temporary appointment, with input from the Working Group. An appointment or election for the vacated office shall be made in accordance with the requirements in Clauses 3.0 and 3.1 at the earliest practical time.


3.3 Removal of Officers


An officer may be removed by approval of two-thirds of the members of the Working Group meeting in Executive Session, or in accordance with the procedures of the Sponsor. Removal of the Chair requires affirmation by the Sponsor. The officer suggested for removal shall be given an opportunity to make a rebuttal prior to the vote on the motion for removal. Removal of the Chair requires notifications to and/or affirmation from the IEEE Computer Society Standards Activities Board Standards Committee


3.4 Responsibilities of Working Group Officers


When carrying out the duties of an officer described in IEEE’s policies and procedures, officers of the Working Group:


  1. shall not act:
    • in bad faith;
    • to the detriment of IEEE-SA;
    • to further the interest of any party outside IEEE over the interest of IEEE; or
    • in a manner that is inconsistent with the purposes or objectives of IEEE; and
  2. shall use reasonable efforts to ensure that participants of the Working Group conduct themselves in accordance with applicable policies and procedures including, but not limited to, the IEEE-SA Standards Board Bylaws clause on “Participation in IEEE standards development.” (See Clause 1.2.)


The officers of the Working Group shall manage the day-to-day operations of the Working Group. The officers are responsible for implementing the decisions of the Working Group and managing the activities that result from those decisions.


3.4.1 Chair


The responsibilities of the Chair or his or her designee shall include


  1. Leading the activity according to all of the relevant policies and procedures.
  2. Forming study groups, as necessary.
  3. Being objective.
  4. Entertaining motions, but not making motions.
  5. Not biasing discussions.
  6. Delegating necessary functions.
  7. Ensuring that all parties have the opportunity to express their views.
  8. Setting goals and deadlines and adhering to them.
  9. Being knowledgeable in IEEE standards processes and parliamentary procedures and ensuring that the processes and procedures are followed.
  10. Seeking consensus as a means of resolving issues.
  11. Prioritizing work to best serve the Working Group and its goals.
  12. Complying with the Chair’s responsibility with respect to the IEEE-SA Intellectual Property Policies, including but not limited to the IEEE-SA Patent Policy (see “Patents” Clause 6 of IEEE-SA Standards Board Bylaws and “Call for patents” Clause 6.3.2 of IEEE-SA Standards Board Operations Manual) and Copyright (see “Copyright” Clause 7 of IEEE-SA Standards Board Bylaws and Clause 6.1 of the IEEE-SA Standards Board Operations Manual).
  13. Fulfilling any financial reporting requirements of the IEEE, in the absence of a Treasurer.
  14. Participating as needed in meetings of the Sponsor to represent the Working Group.
  15. Being familiar with training materials available through IEEE Standards Development Online.
  16. Notifying IEEE SASB of any officer election/appointment, removal, and changes in status.


3.4.2 Vice-Chair


The responsibilities of the Vice-Chair shall include:


  1. Carrying out the Chair’s duties if the Chair is temporarily unable to do so or chooses to recuse himself or herself (e.g., to give a technical opinion) or chooses to delegate specific duties.
  2. Being knowledgeable in IEEE standards processes and parliamentary procedures and assisting the Chair in ensuring that the processes and procedures are followed.
  3. Being familiar with training materials available through IEEE Standards Development Online.


3.4.3 Secretary


The responsibilities of the Secretary include:


  1. Scheduling meetings in coordination with the Chair and distributing meeting notices.
  2. Distributing meeting agenda (as per Clause 6.0). Notification of the potential for action shall be included on any distributed agendas for meetings.
  3. Recording minutes of each meeting according to Clause 6.4 and IEEE guidelines (see, and publishing them within 60 calendar days of the end of the meeting.
  4. Creating and maintaining the membership roster, referred to in Clause 4.5, and submitting it to the Sponsor (or SCC) Secretary annually.
  5. Being responsible for the management and distribution of Working Group documentation.
  6. Maintaining lists of unresolved issues, action items, and assignments.
  7. Recording attendance of all attendees.
  8. Maintaining a current list of the names of the voting members and distributing it to the members upon request.
  9. Forwarding all changes to the roster of voting members to the Chair.
  10. The Secretary shall maintain the attendance record [and responses to letter ballots] for all members on the roster and display or announce the status of voting and non-voting members at the start of each meeting.
  11. Being familiar with training materials available through IEEE Standards Development Online.


3.4.4 Treasurer


Not applicable.


4.0 Working Group Membership


4.1 Obtaining Membership in a Working Group


Working Group participation and membership is on an individual basis (i.e., regardless of affiliation).


A non-member becomes a non-voting member by attending one meeting, providing contact and affiliation information needed for the roster (see Clause 4.5), and requesting membership from the Chair.


A non-voting member becomes a voting member by attending 2 of the last 4 meetings and upon sending a request for voting status to the Working Group Chair prior to the next meeting.


The member’s voting status will be effective at the start of the next meeting.


Voting membership is granted to those participants attending the first meeting of a newly chartered Working Group upon their request to the Working Group Chair.


4.2 Attendance at Meetings


A participant shall attend at least 50% of a meeting’s duration as stated in the approved agenda for that attendance to count towards gaining or maintaining voting membership.


This is called “credited attendance” in order to distinguish it from attendance that does not satisfy these criteria.


Attendance at a meeting via teleconferencing or electronic means, e.g., Internet conferencing, shall count towards the attendance requirements.


4.3 Retaining Membership and Voting Membership


4.3.1 By Attendance


A non-voting member becomes a non-member by not attending any of the last 4 meetings.


A voting member who has not attended at least 2 of the last 4 meetings becomes a non-voting member.


The Chair may “specially maintain” a member’s voting status that would otherwise be lost by these attendance rules. Reasons for such an action can include consideration of personal hardship, medical emergency, or outstanding contributions.


The Secretary will maintain the attendance record for all members on the roster and display or announce the status of voting and non-voting members at the start of each meeting.


4.3.2 Response to Working Group Letter Ballots


A voting member’s obligation to respond to Working Group letter ballots (in which they are eligible to vote) is in addition to their obligation to attend Working Group meetings.


A voting member that fails to return 2 of the last 4 Working Group letter ballots in which they are eligible to vote becomes a non-voting member. [Their existing attendances do not count towards regaining voting membership.]


If a Working Group letter ballot closes within 14 days of a meeting, any changes to voting membership resulting from the ballot shall be delayed until after the meeting.


The chair may “specially maintain” a member’s voting status that would otherwise be lost by this WG letter ballot response rule. Reasons for such an action can include consideration of personal hardship, medical emergency, or outstanding contributions.


4.4 Review of Membership


The procedures for obtaining and retaining membership are specified in terms of meeting attendances, and Working Group letter ballots.


The Chair and Secretary should apply these procedures in a timely fashion, evaluating membership status in time for the next meeting or Working Group letter ballot.


The Chair shall, at least annually, review the status of any members that are specially maintained to validate that the reasons are still applicable.


 4.5 Working Group Membership Roster



A Working Group roster is a vital aspect of standards development. It serves as a record of voting members and members in the Working Group and is an initial tool if an issue of indemnification arises during the process of standards development.


The Secretary shall make reasonable efforts to maintain a current Working Group roster. The roster shall include at least the following:

  1. Title of the Sponsor and its designation.
  2. Title of the Working Group and its designation.
  3. Officers: Chair, Vice-Chair, Secretary.
  4. Members: for each, include name, email address, affiliation, and status (e.g., voting member, non-member, etc.).


All Working Group members are required to review their information contained in the roster during or following each meeting they attend. If a Working Group meets only virtually, it shall determine a schedule to check the accuracy of the roster periodically.


A copy of the Working Group roster shall be supplied to the IEEE-SA at least annually by a Working Group officer or designee. Due to privacy concerns, the roster shall not be distributed, except to the IEEE-SA staff, IEEE-SA Board of Governors and IEEE-SA Standards Board, unless everybody on the roster has submitted their written approval for such distribution.


4.6 Working Group Membership Public List


A Working Group officer or designee shall maintain a current and accurate membership list. The membership list can be posted on the Working Group web site and can be publicly distributed.

The membership list shall be limited to the following:


  1. Title of the Working Group and its designation.
  2. Scope of the Working Group.
  3. Officers: Chair, Vice-Chair, Secretary.
  4. Members: for all, name, affiliation.


5.0 Subgroups of the Working Group


The Working Group may, from time to time, form subgroups for the conduct of its business. Voting Membership in the subgroup is granted to any participant of the Working Group. Such formation shall be explicitly noted in the meeting minutes. At the time of formation, the Working Group shall determine the scope and duties delegated to the subgroup, and may decide to allow participation of persons who are not Working Group members and specify the terms and conditions under which they participate in the subgroup. Any changes to its scope and duties will require the approval of the Working Group. Any resolution of a subgroup shall be subject to confirmation by the Working Group.


The Chair of the Working Group shall appoint, and may dismiss, the Chair of the subgroup.


6.0 Meetings


Working Group meetings may be conducted either exclusively in-person or in-person with one or more participants contributing via electronic means, or exclusively via electronic means. Working Group meetings shall be held, as decided by the Working Group, the Chair, or by petition of 15% or more of the voting members, to conduct business, such as making assignments, receiving reports of work, progressing draft standards, resolving differences between subgroups, and considering views and objections from any source. A meeting notice shall be distributed to all members at least 30days in advance of a face-to-face meeting and at least 21 days notice in advance for an electronic (including teleconference) meeting. A meeting agenda (including participation information) shall be distributed to all members at least 10 days in advance of a face-to-face meeting, and at least 5 days in advance for an electronic meeting. (Meetings of subgroups may be held as decided upon by the members or Chair of the subgroup.) Notification of the potential for action shall be included on any distributed agendas for meetings.


While having a balance of all interested parties is not an official requirement for a Working Group, it is a desirable goal. As such, the officers of the Working Group should consider issues of balance and dominance that may arise and discuss them with the Sponsor.


Participants shall be asked to state their employer and affiliation at each Working Group meeting as required by the IEEE-SA SA Standards Board Operations Manual clause on “Disclosure of affiliation”.


All IEEE standards development meetings are open to anyone who has a material interest and wishes to attend. However, some meetings may occur in Executive Session (see Clause 6.2).


6.1 Quorum


A quorum shall be identified before the initiation of Working Group business at a meeting, but if a quorum is not present, actions may be taken subject to confirmation by letter or electronic ballot, as detailed in Clause 7.2, or at the next Working Group meeting. When the voting membership is less than 50 voting members, a quorum shall be defined as a majority of the current total voting membership. When the voting membership is 50 or more voting members, a quorum shall be defined as 10% of the current total voting membership or 26, whichever is greater. Voting members who recuse themselves shall not be counted in the equation to determine whether a quorum exists.

6.2 Executive Session

Meetings to discuss personnel or sensitive business matters (e.g., the negotiation of contracts), or for other appropriate non-public matters (e.g., the receipt of legal advice), may be conducted in Executive Session.

The matters discussed in executive session are confidential, and therefore, attendance at the Executive Session shall be limited to those with governance authority, outside advisors (e.g., lawyers or consultants) where necessary to provide professional guidance, and select IEEE-SA staff who may have information or a perspective relevant to the subject matter discussed in Executive Session. An individual may be invited to join for a portion of the discussion and then excused at the appropriate time. In each case, except as authorized by the Working Group, participants in an Executive Session are prohibited from discussing or disclosing any information presented and discussed during such Executive Session to a third party or other person not present during the Executive Session, and shall not continue to discuss such matters after the Executive Session has adjourned.

Executive Sessions should be conducted face-to-face (in person) to provide the greatest assurance that the content of such Executive Sessions will be kept confidential. However, when necessary, Executive Sessions may include participants who participate by teleconference provided such persons agree not to disclose any information so discussed, and agree that they will participate in such conference in a manner that does not result in third parties gaining access to such discussions or information.


6.3 Meeting Fees


The Working Group, or meeting host, may charge a meeting fee to cover services needed for the conduct of the meeting. The fee shall not be used to restrict participation by any interested parties.

The Working Group Officers shall set the meeting fees in consultation with those planning a particular meeting.

Everyone who attends a meeting, except for those specifically excluded due to hardship, shall pay the meeting fee.


6.4 Minutes


The minutes shall concisely record the essential business of the Working Group, including the following items at a minimum:

  1. Name of group
  2. Date and location of meeting
  3. Officer presiding, including the name of the secretary who wrote the minutes
  4. Meeting participants, including affiliation, and voting member status at the end of the meeting
  5. Call to order, Chair’s remarks
  6. Reminders of IEEE policies, such as Patent policy and Copyright policy
  7. The fact that a Call for Patents occurred and any responses made to such Call
  8. Approval of minutes of previous meeting
  9. Approval of agenda
  10. Technical topics
    • Brief summary of discussion and conclusions
    • Motions exactly as they are stated, including the names of mover and seconder and the outcome of each motion
  11. Action items
  12. Items reported out of executive session
  13. Recesses and time of final adjournment
  14. Next meeting – date, time, and location

All submissions, presentations, and reports considered during the meeting by the Committee/WG shall be referenced in the minutes, identifying the source of the submission. URLs should be provided where possible.

The following shall not be recorded in minutes:

  • Transcriptions of detailed discussions
  • Attributions of comments to specific participants


7.0. Voting


7.1 Approval of an Action


Approval of an action requires approval by a majority (or two-thirds) vote as specified below in 7.1.1 (majority) and 7.1.2 (two-thirds). The “majority, two-thirds vote” is defined as either:


  1. At a meeting (including teleconferences) where quorum has been established, a vote carried by majority (or two-thirds) approval of the votes cast (i.e., Approve or Do Not Approve votes, excluding abstentions) by the voting members in attendance.
  2. By electronic means (including email), a vote carried by majority (or two-thirds) of the votes cast (i.e., Approve or Do Not Approve votes, excluding abstentions), provided a majority of all the voting members of the Working Group responded.


7.1.1 Actions Requiring Approval by a Majority Vote


The following actions require approval by a majority vote:


  1. Formation or modification of a subgroup, including its procedures, scope, and duties
  2. Disbandment of subgroups
  3. Approval of minutes.


7.1.2 Actions Requiring Approval by a Two-thirds Vote

The following actions require approval by a two-thirds vote:


  1. Approval of change of the Working Group scope *
  2. Establishment of fees, if necessary
  3. Approval to move the draft standards to the Sponsor for IEEE Standards Sponsor ballot*


*Items a) and c) also require approval of the Sponsor [or Responsible Subcommittee] to take effect.


7.2 Voting Between Meetings


The Working Group Chair shall conduct votes authorized by the Working Group in a timely fashion. The Working Group Chair may conduct votes between meetings by the use of a letter or electronic ballot. If such actions are to be taken, they shall follow the rules of IEEE Bylaw I-300.4(4).


8.0 Communications


Formal inquiries relating to the Working Group should be directed to the Chair and recorded by the Secretary. All replies to such inquiries shall be made through the Chair. These communications shall make it clear that they are responses from the Working Group. Communications shall be in compliance with the Sponsor’s communication requirements.


9.0 Appeals


Any person dissatisfied with a technical decision shall follow the approved procedures for providing technical input to the Working Group, including but not limited to presenting the concern to the Working Group, and making a technical comment during the applicable comment submission and/or balloting period.


Procedural concerns within the Working Group shall first be presented to the Working Group Chair for resolution. If the procedural concern is not resolved after presentation to the Chair, the concern can be brought to the Sponsor for resolution.