Deprecated: Function libxml_disable_entity_loader() is deprecated since 8.0, as external entity loading is disabled by default in /application/src/init.php on line 167
Deprecated: Return type of Qrequest::offsetExists($offset) should either be compatible with ArrayAccess::offsetExists(mixed $offset): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 22
Deprecated: Return type of & Qrequest::offsetGet($offset) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 25
Deprecated: Return type of Qrequest::offsetSet($offset, $value) should either be compatible with ArrayAccess::offsetSet(mixed $offset, mixed $value): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 31
Deprecated: Return type of Qrequest::offsetUnset($offset) should either be compatible with ArrayAccess::offsetUnset(mixed $offset): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 35
Deprecated: Return type of Qrequest::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 38
Deprecated: Return type of Qrequest::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 88
Deprecated: Return type of Qrequest::jsonSerialize() should either be compatible with JsonSerializable::jsonSerialize(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /application/lib/qrequest.php on line 91
Warning: Cannot modify header information - headers already sent by (output started at /application/src/init.php:167) in /application/src/initweb.php on line 45
Warning: session_name(): Session name cannot be changed after headers have already been sent in /application/lib/redirect.php on line 46
Warning: session_cache_limiter(): Session cache limiter cannot be changed after headers have already been sent in /application/lib/redirect.php on line 47
Warning: session_set_cookie_params(): Session cookie parameters cannot be changed after headers have already been sent in /application/lib/redirect.php on line 60
Deprecated: Creation of dynamic property Qrequest::$t is deprecated in /application/lib/qrequest.php on line 42 Help - Formulas - Netdev 0x18
Program committee members and administrators can search and display formulas
that calculate properties of paper scores—for instance, the
standard deviation of papers’ Overall merit scores, or average Overall
merit among reviewers with high Reviewer expertise.
To display a formula, use a search term such as “show:var(OveMer)” (show
the variance in Overall merit scores, along with statistics for all papers).
You can also graph formulas.
To search for a formula, use a search term such as “formula:var(OveMer)>0.5”
(select papers with variance in Overall merit greater than 0.5).
Or save formulas using Search > View options > Edit formulas.
Formulas use a familiar expression language.
For example, this computes the sum of the squares of the overall merit scores:
sum(OveMer*OveMer)
This calculates an average of overall merit scores, weighted by expertise
(high-expertise reviews are given slightly more weight):
wavg(OveMer, RevExp >= 4 ? 1 : 0.8)
And there are many variations. This version gives more weight to PC
reviewers with the “#heavy” tag:
wavg(OveMer, re:#heavy + 1)
(“re:#heavy + 1” equals 2 for #heavy reviews and 1 for others.)
Formulas work better for numeric scores, but you can use them for letter
scores too. HotCRP uses alphabetical order for letter scores, so the “min” of
scores A, B, and D is A. For instance:
count(confidence=X)
Expressions
Formula expressions are built from the following parts:
Arithmetic
2
Numbers
true, false
Booleans
e + e, e - e
Addition, subtraction
e * e, e / e, e % e
Multiplication, division, remainder
e ** e
Exponentiation
e == e, e != e, e < e, e > e, e <= e, e >= e
Comparisons
!e
Logical not
e1 && e2
Logical and (returns e1 if e1 is false, otherwise returns e2)
e1 || e2
Logical or (returns e1 if e1 is true, otherwise returns e2)
test ? iftrue : iffalse
If-then-else operator
(e)
Parentheses
greatest(e, e, ...)
Maximum
least(e, e, ...)
Minimum
log(e)
Natural logarithm
log(e, b)
Log to the base b
round(e[, m])
Round to the nearest multiple of m
null
The null value
Submission properties
pid
Paper ID
au
Number of authors
au:pc
Number of PC authors
au:text
Number of authors matching text
Tags
#tagname
True if this paper has tag tagname
tagval:tagname
The value of tag tagname, or null if this paper doesn’t have that tag
Scores
overall-merit
This review’s Overall merit score
Only completed reviews are considered.
OveMer
Abbreviations also accepted
OveMer:external
Overall merit for external reviews, null for other reviews
OveMer:R2
Overall merit for round R2 reviews, null for other reviews
Submitted reviews
re:type
Review type
re:round
Review round
re:auwords
Review word count (author-visible fields only)
re:primary
True for primary reviews
re:secondary
True for secondary reviews
re:external
True for external reviews
re:pc
True for PC reviews
re:sylvia
True if reviewer matches “sylvia”
Review preferences
pref
Review preference
prefexp
Predicted expertise
Aggregate functions
Aggregate functions calculate a
value based on all of a paper’s submitted reviews and/or review preferences.
For instance, “max(OveMer)” would return the maximum Overall merit score
assigned to a paper.
An aggregate function’s argument is calculated once per visible review
or preference.
For instance, “max(OveMer/RevExp)” calculates the maximum value of
“OveMer/RevExp” for any review, whereas
“max(OveMer)/max(RevExp)” divides the maximum overall merit by the
maximum reviewer expertise.
The top-level value of a formula expression cannot be a raw review score
or preference.
Use an aggregate function to calculate a property over all review scores.