Most program committees do not really evaluate submissions. They evaluate their own level of energy. Read 140 abstracts across three evenings and you may score more strictly on the first evening than on the third, reward polished writing over strong ideas, and remember the talk that appeared near the top of the list. That is human. But it is also a process problem, and it can be fixed.
A call for papers has two jobs: fill the program and make the program better. Most conferences manage the first. Many struggle with the second because scoring and selection happen in a single spreadsheet with ten columns and four opinions.
This article shows how to build a review process that remains consistent with 200 submissions, where AI can take over preparation work, and where the final decision needs to stay with your committee.
The problem is rarely volume. It is comparability.
Two reviewers, one abstract, two completely different scores: this happens as soon as the criteria are not clearly defined. “I like it” is not a criterion. “Fits the program” is not either unless everyone agrees on what the program actually needs.
A useful scoring sheet for the program committee has five or six criteria, each with a short scale and an anchor example for the highest and lowest score:
- Audience relevance: Does the topic solve a problem participants actually have?
- Novelty: Is there something new, or is known material simply being presented again?
- Practical relevance: Are there concrete cases, figures, or experiences rather than theory alone?
- Format fit: Does the proposal fit a keynote, session, workshop, or panel?
- Speaker track record: Is there a reference, video, or previous appearance?
- Contribution to diversity: Does the proposal add something to the program, or are there already three similar sessions?
The last criterion is often forgotten. It cannot really be judged submission by submission. It only becomes clear in comparison, which is why it belongs in a second round.
Two review rounds instead of one
A process that works well for mid-sized conferences looks like this:
Round 1: Individual scoring
Each submission gets at least two independent reviews, without reviewers seeing each other’s scores. This reduces anchoring effects. It also helps to randomize the order of submissions for each reviewer so everyone is not reading the same proposals when they are already tired.
Round 2: Program building
The committee no longer looks at individual scores alone. Instead, it looks at the overall picture: Which topics are duplicated? Which audiences are missing? How many newcomers are in the program? How many repeat speakers? This is where a ranking becomes a program.
An example calculation (assumptions, not measured results): 180 submissions, two reviews per submission, and an average of ten minutes per review. That means 360 reviews and 60 hours of work. Spread across six committee members, that is ten hours per person, a workload that is difficult to handle properly alongside day-to-day work. If an initial screening clearly removes 40 percent of submissions (outside the topic scope, incomplete, or a pure product pitch), the workload falls to around six hours per person.
Where AI can help with the review process
AI works best here as an assistant for preparation, not as the judge. Four use cases are particularly practical:
Check completeness and formal requirements
A language model can check whether each submission includes a title, abstract, learning objectives, format, and speaker details, and flag missing information for a follow-up request. This reduces manual chasing.
Identify product pitches
An abstract that is essentially a sales presentation can be flagged using clear signals: a product name in the title, no customer examples, or no meaningful insight without a purchase. The flag is a signal for the committee, not an automatic rejection.
Cluster topics
AI can turn 180 abstracts into twelve topic clusters. You immediately see, for example, that 31 submissions focus on AI in marketing while only four address sustainability, allowing the committee to make a conscious decision about the balance.
Create short summaries for review
Instead of starting with 400-word abstracts, the committee can first read a three-sentence summary with a link to the original. This speeds up the first round, but the original submission must remain available at all times.
What AI should not take over: the final score. A model evaluates language patterns, not relevance to your community. It can also favor polished writing over rougher submissions that contain stronger ideas. If you use AI, disclose this in the call for papers and process personal data only with tools covered by an appropriate data processing agreement.
Three biases you can actively reduce
Name bias
Well-known speakers can receive better scores even when their abstract is weaker. One countermeasure is a blind first round: show the title and abstract, but hide the name and company.
Language bias
Native speakers or especially fluent writing can appear more competent. Ask the committee to focus deliberately on content, structure, and examples.
Similarity bias
People tend to rate topics higher when they resemble their own work. Topic clustering makes it visible whether the program has become a reflection of the committee itself.
A simple test after round two: How many sessions come from people who were already on stage the previous year? How many come from organizations that are already sponsors? There is no correct number, but there should be a number you know and can explain.
Rejections and acceptances: the part that shapes your reputation
A call for papers is also brand management. Send 120 rejection emails and you are creating 120 impressions of your conference. Three rules help:
- Set a firm deadline: State in the call for papers when acceptances and rejections will be sent, and stick to the date.
- Give one line of reasoning: One or two sentences explaining the decision take little time and increase the chance that rejected speakers will apply again next year.
- Include next steps in acceptances: Give the acceptance deadline, deadlines for slides and bio, a contact person, and information about compensation or travel costs.
AI can prepare text blocks for acceptances and rejections that you then personalize. But a rejection to someone who spent three months working on a proposal should still be read and approved by a human.
Where eventpage.ai fits in
The review itself can continue to run through the process your team already uses. Before and after the review, the organization has other tasks to manage: the event page with a clear description of the program framework, participant registration, ticketing, and the agenda where accepted sessions later appear.
With eventpage.ai, the event page, registration, and ticketing come together in one place. For the organization, OrganizationKIT creates an AI briefing with tasks that can support coordination within the committee, while CommunicationKIT helps prepare communications for speakers and participants. Which functions are included in your plan is shown in the current pricing and feature overview.
Invitation Management around the call for papers
The process does not end when the review is complete. Acceptances, rejections, and next steps need to reach selected and rejected speakers clearly. This is where Invitation Management becomes part of a structured workflow: the program committee’s decision needs to be turned into clear communication covering deadlines, slides, speaker bio, contact details, and information about compensation or travel costs.
Especially with many submissions, a clear separation helps: first make submissions comparable and curate the program, then communicate the decisions in a structured way. This keeps the selection process understandable and makes speaker communication the next clear step in the conference workflow.
Conclusion and next step
A strong program does not come from more opinions. It comes from comparable scores and a second look at the overall picture. AI can shorten the preparation work and reveal patterns, but the decision stays with the committee.
For getting started, we have put together a scoring sheet with six criteria, a scale, and anchor examples.
If you want to see how event pages, registration, and ticketing work together with eventpage.ai, you can start for free.
