SDH in video games

SDH in video games

Hello everyone.
It’s Tomás Costal here.
Welcome to this critical analysis
of accessibility in video games.
Today, we will discuss the role
of SDH subtitles
or subtitles for the deaf
and hard of hearing
sometimes referred to
as closed captions or CC.
We will start with a general outline
of what common practice indicates
a subtitle should be
the essential conventions
to be followed
and the changes that are needed
to reduce incongruity
and extreme variance
in how SDH are created
and rendered
in entertainment products
and a diversity of media outlets.
But, first things first.
Subtitling requires condensation
of the original message
linguistic expertise and,
last but not least
dexterity in audiovisual translation.
Translation, in this context
does not exclusively mean
from language A, such as English
to language B, such as Spanish.
It applies to different varieties
of the same language.
When we proceed
from a source to a target language
as in the case of English to Spanish
or vice versa
that is interlinguistic translation.
On the other hand, when we work
within the boundaries of a language
whichever it may be
the term we use
is intralinguistic translation.
However, in today’s analysis
we will deal much more
with the way norms and conventions
may help all stakeholders
industry and final users
to identify core needs
and requirements
which contribute to the usability
and accessibility
of audiovisual
multimedia productions.
Subtitles in other media
share a similar layout.
As consumers of audiovisual products
we are used to two lines
at the bottom of the screen
centred, appearing
when the utterance starts
and disappearing when it ends.
In other words,
subtitles must be easily readable
and synchronised with the actions
which are taking place.
If subtitles go too fast,
we won’t be able to read them
and if they go too slowly,
we will read them more than once
and this will cause us discomfort.
Badly edited subtitles, therefore,
are cumbersome for the viewer
and bad choices made
during the editing phase
are due to poor quality assessment
lack of attention to detail
the lack of a language department
throughout the production
and development stages
as well as idiosyncratic
rather than well-informed decisions.
In the case of video games
resorting to a norm
with medium-appropriate guidelines
is definitely the way to go.
Let’s see a few case studies
observe what performance looks like
apply our critical analysis
and summarise our findings
to advance a proposal of a norm
for SDH in video games.
To put it another way,
we will start with the particular
and build up to the general.
Each case study comes in three parts.
Firstly, we describe the video game
and indicate the YouTube source
of our cutscenes.
Secondly, we will go over
the video clips and screenshots
and make observations concerning
the video game’s so-called subtitles.
In some cases,
these “subtitles” are hardcoded.
In other words,
they cannot be removed
even if the player wants to.
In some other cases,
they were activated for this study
but feature in the video game
as optional.
Thirdly, we will indicate
which aspects are acceptable
and should become part
of our tentative SDH norm
and which ones should be suppressed,
omitted or modified
to avoid obscurity, errors
and end user dissatisfaction.
Let’s get to it then.
Case study one.
Back to the Future: The Game.
Produced by Telltale Games
and released in 2010
for several platforms.
The cutscenes, or game sections
where interaction is minimal
and watching prevails
over doing things with the controller
were obtained
from Gamer’s Little Playground.
All references to the games
and source materials
are included in the description.
Let’s see the clip
from this graphic adventure
based on the film
starring Michael J. Fox
and pay attention to three points.
Number one: where are the subtitles
positioned on the screen?
Number two: do the lines appear
all at once?
that is to say,
are they pop-on subtitles?
or do they appear progressively?
Roll-up subtitles.
Finally, number three.
Is synchronisation compelling enough
between the mouth movements,
the dialogue and the subtitle text?
The clip is 45 seconds long
so feel free to take notes
and stop whenever necessary.
That’s our first clip.
Notice that synchrony
leaves much to be desired
both in terms of voices and text.
Also, the subtitle lines roll out
rather than pop in.
This takes more time
and is generally more distracting
for the viewer and the player.
Finally, subtitles change position
depending on the amount of text
that needs to be shown.
With the help of these screenshots
we may see the problems
of roll-out subtitles.
Here, the centre of the screen
contains two lines
which appear superimposed
over the character’s face.
Taken to the extreme,
this is exasperating for the players
and hinder their interaction.
In addition,
the lines are extremely long
and incorrectly divided.
Line division
should not be arbitrary.
Syntactic coherence
is not simply desirable
but compulsory in subtitling
especially when we think
that hundreds of thousands
if not millions or tens of millions
are exposed to the linguistic models
and practices
of a given video game.
As we can see in this screenshot
quality assurance focused much more
on pure orthography
than subtitle sequencing
and readability.
Here’s our final verdict
for the first case study.
General quality is rather poor
and the reasons are stated below.
Roll-out subtitles are inefficient
in terms of understanding
line length should be restricted
in all cases
syntactic coherence should be revised
and under no circumstances
should we cover people’s mouths
characters’ faces
and the central action taking place
with text.
Let’s move on to case study two.
This is a puzzle adventure game
entitled Catherine
developed by Atlus
and first released in 2011.
Our source for cutscenes
was UPlayNetwork.
There’s a link in the description.
Here’s the first of two video clips.
It is time to analyse elements
other than dialogue or interaction.
What we call paralinguistic elements.
Sounds we produce
and attach meaning to
although this meaning varies
between cultures.
They cannot easily be reflected
as textual elements.
We will talk about them
after the clip.
That’s our clip.
What about the ellipsis
and exclamation point
as a paralinguistic descriptor?
In this case, the character’s
almost psychotic countenance
accompanied by an increase
in sound effects.
Does it really add anything?
Indeed, does it help the reader
to grasp the content better?
Before giving a final answer
remember that this game
was dubbed into English
with the original in Japanese.
After all, we are dealing
with cultural differences
as we said before.
Apart from that, a line of text
in these subtitles
emulated the sound
of the male character’s swallowing
Asterisk, glug, asterisk
repeated twice.
This is the game’s interpretation
of a closed caption
idiosyncratic as it may be.
If you were deaf or hard of hearing
you would not have access
to those noises, sounds
soundtracks, scores, accents
or speakers’ moods and attitudes
which can only be perceived
through the ear.
SDH subtitles
enrich traditional subtitles
by taking this requirement
into account
and with paralinguistic information
not immediately obvious
from what is on the screen.
Perhaps the asterisk, glug, asterisk
repeated twice
was omitted because the character
is seen drinking
from his china cup of tea.
As for the ellipsis,
exclamation point
we discussed earlier
that the deaf and hard of hearing
as anyone else
are not aided in any way
by the inclusion of this subtitle.
Let’s continue with the second clip.
We know that the lines,
in this game too, are very long.
Mostly because they cover the screen
from left to right
and the action takes place
towards the centre.
What about shot changes?
If the shot changes,
but the subtitle line remains
the players may think
that the text is not the same
and direct their eyes
to the beginning of the line.
Quick exchanges make this situation
inevitable at times
but very often, as in this clip
such situation may be avoided.
Have a look and you be the judge.
Focus on the part
when the man in the sunglasses
talks about sleep and sheep
and count the shot changes.
Notice there are four changes
simultaneously with the utterance.
This practice is discouraged
in film and on television.
We would argue that video games
either in cutscenes
or interactive sequences
should keep this aspect in mind.
Perhaps even more so
than in other media.
A question we will revisit later
is character identification
or utterance tagging.
In other words, making explicit
through text or colour
who the actual speaker is.
In the previous scene,
the four men, the waitress
and the man in sunglasses intervene
at different times
but for the deaf or hard of hearing
it may be challenging
to make this out at all times.
Finally, let us analyse
a few screenshots more closely.
Here, the lines are too long.
In any case, the upper limit
should be set
within the same margins
as television or film.
That is, from 35 to 40 characters.
Preferably, under 37 characters,
including spaces and punctuation.
This would lead to an increase
in the total number of lines
but would facilitate normalisation
and reduce the number of exceptions
both subtitlers
and programmers would face.
Here the two lines
are extremely different in length
and though they endeavour to maintain
syntactic coherence
we would advise condensation
of the message
together with further subdivision.
Here, the only element
that is subtitled
is the language produced orally.
The audio track.
However, if we were translating
we would need to resort
to some specific translation strategy
to deal with the other two messages
present as text on screen.
“Stray sheep”, on the background,
as if it were the name of the bar
which would be highly plot-relevant.
And then we have
the “Italian stallion” text
on the shoulder jacket
perhaps as an inside joke
concerning the protagonist’s
expensive taste in fashion.
Here, we can see a combination
of paralinguistic
and linguistic elements
in the same line.
Once again, a subtitling norm
oriented towards SDH in video games
would need to specify the criteria
for the former and the latter.
Why the asterisks? we might ask.
Why not capital letters instead?
Or a different colour.
Here, apart from the question
of asterisk, munch, asterisk
the identification problem persists.
Which of the young men is talking?
When lip movements are unclear
we would need an alternative solution
which does not clash
with what we have stated so far
about paralinguistic elements.
Of course, this single line
is extremely long.
Condensation of the message
would still be required.
Finally, our critical verdict.
These subtitles would be acceptable
with certain alterations.
Paralinguistic elements would need
to be normalised
line length should be restricted
shot changes respected
and speaker identification is needed.
And with this we reach
case study number three.
Anarchy Reigns, a beat ’em up
2012 video game
developed by Platinum Games.
The YouTube source for our cutscenes
is Rabid Retrospect Games.
There’s a link in the description.
We should mention that budget costs
in the design and voice acting stages
are reduced when cutscenes decrease
in cinematic quality.
In this case, the dialogue
is quite loosely synchronised
and the character portraits
are almost static.
This happens
for almost a third of the game.
Before you watch,
consider how much text is shown
the way it is sequenced and divided
whether it is syntactically coherent,
faithful to what is uttered
or condensed.
And if it pays heed to the needs
of SDH audiences.
Durga and Led’s exchange
identifies the speakers by name.
However, the colour is the same
for both speakers
and easy to confuse
with the background.
The font is tiny and hard to read
and does not reflect
paralinguistic elements in any way.
Durga’s laugh is lost
for SDH players.
And Led’s irritation
should have been conveyed
using didascalic mood indicators
such as (IRRITATED)
and then the text.
The following screenshots
reinforce our initial argument.
Douglas’ three-liner
is incorrectly divided
and for the first time in the game
uses three lines instead of two.
None of them is used for tagging,
describing sounds or reflecting moods
which constitutes a viable exception
to the two-line rule.
Again, the font is too small
and the colour clashes
with the background hue.
In this light, the long one-liner
should have been divided in two parts
the second one starting
in “to such criminal activity”.
Although the font is hard to read,
the colour is clearly visible.
However, a black box around it
would have made it stand out more.
Here, the line division is adequate
but the lines unnecessarily long
which might explain
why the font type is so small.
Finally, our verdict
for the third case study.
Again, the subtitles
would be considered acceptable
with certain alterations,
but they are definitely not SDH.
The font size
should urgently be increased.
The colour clash avoided.
Different colours should be assigned
to different characters.
Paralinguistic elements
must be included.
And the number of characters
needs a limit.
Now comes case study number four,
Deadpool the video game
released in 2013
by High Moon Studios.
The source for this beat ’em up
is Red’s 3rd Dimension Gaming.
This time, you may proceed
to analyse the first clip
and put yourself in the shoes
of a deaf player.
What is missing? There we go.
We could argue that Deadpool
is no common video game
and fourth wall breaks
are more frequent
than the graphic novel fanatic
would ever have imagined.
Nevertheless, certain aspects
would benefit accessibility.
Firstly, we did hear the crickets
but the deaf person
wouldn’t see that in the subtitle.
This decision needs alteration.
Also, mood indicators
would be highly advisable
as one of the characters,
called Wade, or Deadpool
or, to simplify it even more,
the masked man
together with Cable, half man,
half machine
both interventions range
from the hilarious to the raging mad.
The music is also lost
if closed captions are not included.
And, last but not least,
the “What you talkin’ ’bout, Summers”
features a comical take
on a specific North American accent.
What to do with accents
in traditional or SDH subtitles
both intra and interlinguistically
when we are translating
is a very fertile line of research
but still unresolved.
This second Deadpool clip
has been selected as a sample
of inappropriately divided subtitles
lengthy one-liners
and the high speeds players endure.
As for the sound effects
and their description
what is seen on the screen
like the energy blasts Sinister uses
to attack the X-Men
in conjunction
with controller vibration
would seem to be sufficient.
In our selection of screenshots
from the video game cutscenes
we can see the idiosyncratic nature
of onomatopoeic
and paralinguistic elements.
In spite of this observation,
we should note that subtitling norms
even more so in initial stages
when they stand as repositories
of best practices
are never designed
to stifle creativity
or limit the versatile nature
of the medium.
Far from it.
The intention is to gather evidence
of what happens in the industry
and facilitate the procedure
as far as possible.
Norms, for instance, help prevent
incongruities such as these
where chevrons and asterisks
are used for sound description.
In addition, they state clearly
and succinctly
what the sound description,
character identification
noise tagging
or paralinguistic element inclusion
would comprise exactly.
In Deadpool, we had the chance
to see many of the issues
the norm should consider
from line division, size, colour,
speed and synchrony
to description of noise, sound, mood
character identification,
accent inclusion
and orthotypography.
Here’s our verdict.
The subtitle is acceptable
with changes.
The paralinguistic elements
need normalisation and synchrony.
The speed should be reduced
sometimes it’s difficult
to follow the subtitle.
And, more importantly
we should distinguish
between characters, noises, sounds
moods and accents
according to coherent criteria.
From this point onwards,
you will watch the clips first
produce your own assessment
and compare it with the comments
and the screenshots
which help reinforce
the main points of contention.
Case study number five.
Alien: Isolation (2014)
a survival horror video game
developed by Creative Assembly.
The cutscenes were excerpted
from Gamer’s Little Playground.
There we go. First clip.
Consider line length, size,
colour and speed
as the main variables
for your subtitle judgement.
Now it’s time for clip number two.
Use the same parameters.
And now, the assessment.
The general quality is rather poor.
The font is unreadable.
The text is excessive.
And these are clearly not subtitles
but simply a fragmentary screenplay
or rather, a literal transcript.
Let us have another go
with Castlevania Lords of Shadow 2
by Mercury Steam.
An action adventure game
released back in 2014.
There is a link to the cutscenes
in the description.
Once again, consider line length
size, colour, speed, orthography
typography and synchrony
as your variables.
Pay special attention
to syntactic coherence the divisions.
The clip lasts 30 seconds
and is followed by seven screenshots.
You will be given time to see them
but you may pause if necessary.
Have you got it already?
Here’s our verdict.
We find the subtitles acceptable,
but they need changes.
For instance, the question of accents
is not yet resolved.
Line divisions are wrong
most of the time.
There are too many lines,
although not excessive in length.
The amount of on screen text
is at times overwhelming
although, we should insist,
this is a very long video game
so the judgement is rather partial.
In conclusion, the subtitles are fine
for a traditional subtitle
but we were looking for SDH subtitles
and that is what we were promised
when the video game said
that it was SDH accessible.
Alright then.
It is now time for Halo 5,
our seventh case study.
This is a triple A
first person shooter
developed by 343 Industries
and released in 2015.
Watch the 30-second clip
and decide
if there’s anything different
about how new subtitle lines
are presented to the player.
In addition, reflect upon the choice
of character identification system
the use of colour
and whether any information is lost
for the SDH player.
The disappearance
of the upper subtitle lines
while the lower ones remain
the proliferation of text
in the lower half of the screen
the use of the same colour
for all speakers
and the lack of differentiation
between characters present on screen
and those who are not
such as the “hundreds of AI voices”
referred to using italics
as in television or film
increases the level of confusion
for the player.
The lines are long. The text is tiny.
And the colour unclear
in certain environments
as we appreciate
in the selected screenshots.
The verdict.
Acceptable with modifications.
It needs further condensation.
It needs to reconsider the system
of subtitle presentation
for the player
and the deaf player in particular.
Finally, it must include
more SDH nuances
if it wants to be considered
an SDH subtitle.
For example, mood indicators
and sound and noise descriptions.
To conclude this preliminary study
let’s work in the same condition
a deaf player must face
with the premise that players
have English as their mother tongue
instead of sign language
or any other language
which makes audiovisual translation
more complicated
as well as add intersemiotic value
to the transformations necessary.
This first clip is from Catherine
one of the games discussed earlier.
Are the subtitles appropriate?
Have we lost anything on the way?
Are deaf players
in the exact same conditions
as long as the subtitle
is present on the screen?
Once again, you be the judge.
Get ready and sound off.
There we go.
Now it’s time to compare
and contrast.
The sound is on.
What are the differences
between the sound off
and on versions of this clip?
Firstly, the siren is heard first
and seen afterwards
which places the deaf viewer
and player at a disadvantage.
Also, the breaking news are,
inexplicably, not subtitled.
They should have been.
Ambient noise,
eavesdropped conversations in the bar
and the waitress’s words
are not subtitled
when the camera moves
further into the bar
where the young people have
their weekly encounter.
These elements should have featured
in an SDH subtitle.
So we cannot be sure
who’s actually speaking at any point.
For this reason,
this is no SDH subtitle.
In fact, it is an incomplete
traditional subtitle.
Finally, one last try
in the same conditions.
We will use an opening sequence
taken from Halo 5.
No sound first.
There we go.
And now,
with the additional semiotic layer.
Can you notice
any relevant differences?
How many of the differences
could be compensated for
using of non-linguistic resources?
Such as visual elements
and special effects
and controller vibration
following a particular pattern.
Have a look.
As you see, subtitles add very little
to the rich sound landscape
although considering
controller vibration
as part of the SDH subtitling norm
is remarkably attractive
and invites further development.
Our verdict is: not SDH,
it needs alteration.
Given that the particulars
of the SDH norm for video games
will be discussed in depth
in an upcoming presentation
let us take stock
of what we have observed so far.
Which categories should the SDH norm
focus more specifically on?
As for the degree of faithfulness.
Is it a subtitle
or is it a transcript?
They are not the same.
Subtitle type.
Is it traditional or is it SDH?
These contain more information.
As for its normativity.
Is it idiosyncratic?
Does it decide depending on a whim?
Or does it use conventions?
As for the relation
between the sound and the text
we should pay attention to dialogue
music and soundtrack
tagging or speaker identification
text on screen
or in-game instructions
paralinguistic elements
onomatopoeia and its use
and didascalic mood indicators.
The attitudes of the speakers
towards what they are saying.
Lastly, the purely textual level.
What does it comprise
and what should be in a future norm?
Obviously the sequencing
because from the transcript
to the subtitle
the distinction lies
in how the information is channelled
in a more condensed way.
The maximum number of lines
is something to be decided
but two is the ideal.
Line division, as we said,
has to be syntactically coherent.
The shot changes.
Let’s not forget those.
The speed measured in CPS,
or characters per second.
Orthography and typography
as relevant
as in any textual representation.
Capitalisation, emphasis
and the use of italics
controversial even in television
and other formats, such as film.
Here are the last questions
to draw one’s own conclusions.
Question number one.
Where do the inconsistencies lie
when we discuss accessibility and SDH
in the medium of video games?
And question number two.
Would a common framework
really be deleterious
for the creative freedom
or the originality of this industry?
And with this,
feel free to leave any comments.
Thank you for watching
this presentation.
It has been a pleasure.

1 thought on “SDH in video games”

  1. So whats an appropriate use of hyphens vs dashes? For example a person interrupts himself to say a different sentence?

Leave a Reply

Your email address will not be published. Required fields are marked *