epubjs
Version:
Render ePub documents in the browser, across many devices
83 lines (81 loc) • 5.12 kB
HTML
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Talk to Me: Media Overlays</title>
<link rel="stylesheet" type="text/css" href="css/epub.css" />
<meta name="dat-origPath" value="/html/body/section/section[1]" /><link rel="prev" href="./ch03s02_3.html" /><link rel="next" href="./ch03s02_5.html" /></head><body>
<p data-origPath="/html/body/section/section[1]/p[16]">But those concerns aside, that’s all there is to basic text and audio
synchronization. So, as you can now see, no reading system witchcraft was
required to synchronize the text document with its audio track! Instead, the
audio playback is controlled by timestamps that precisely determine how an audio
recording is mapped to the text structure. Whether synchronizing down to the
word or moving through by paragraph, this process doesn’t change.</p>
<p data-origPath="/html/body/section/section[1]/p[17]">To synchronize the first three words “Call me Ishmael” in the first paragraph,
for example, we simply repeat the process of matching element ids and audio
offsets:</p>
<pre class="screen" data-origPath="/html/body/section/section[1]/pre[5]"><par>
<text src="chapter_001.xhtml#c01w00001"/>
<audio
src="audio/mobydick_001_002_melville.mp4"
clipBegin="0:00:29.269"
clipEnd="0:00:29.441"/>
</par>
<par>
<text src="chapter_001.xhtml#c01w00002"/>
<audio
src="audio/mobydick_001_002_melville.mp4"
clipBegin="0:00:29.441"
clipEnd="0:00:29.640"/>
</par>
<par>
<text src="chapter_001.xhtml#c01w00003"/>
<audio
src="audio/mobydick_001_002_melville.mp4"
clipBegin="0:00:29.640"
clipEnd="0:00:30.397"/>
</par></pre>
<p data-origPath="/html/body/section/section[1]/p[18]">You’ll notice each <code class="literal">clipEnd</code> matches the next
element’s <code class="literal">clipBegin</code> here because we have a single
continuous playback track. Finding each of these synchronization points manually
is not so easy, though, as you might imagine.</p>
<p data-origPath="/html/body/section/section[1]/p[19]">Synchronizing to the sentence level, however, means only one synchronization
point is required for all the words the sentence contains, thereby reducing the
time and complexity of the process several magnitudes. The <code class="literal">par</code> is otherwise constructed exactly like the previous example:</p>
<pre class="screen" data-origPath="/html/body/section/section[1]/pre[6]"><par>
<text src="chapter_001.xhtml#c01s0002"/>
<audio
src="audio/mobydick_001_002_melville.mp4"
clipBegin="0:00:30.397"
clipEnd="0:00:44.783"/>
</par></pre>
<p data-origPath="/html/body/section/section[1]/p[20]">The process of creating overlays is only complicated by the time and text
synchronizations involved, as is no doubt becoming clearer. Moving up another
level, paragraph level synchronization reduces the process several more
magnitudes as all the sentences can be skipped. Here’s the single entry we’d
only have to make for the entire 28s second paragraph:</p>
<pre class="screen" data-origPath="/html/body/section/section[1]/pre[7]"><par>
<text src="chapter_001.xhtml#c01p0002"/>
<audio
src="audio/mobydick_001_002_melville.mp4"
clipBegin="0:01:46.450"
clipEnd="0:02:14.138"/>
</par></pre>
<p data-origPath="/html/body/section/section[1]/p[21]">The complexity isn’t only limited to the number of entries and finding the audio
points, however, otherwise technology would easily overcome the problem.
Narrating at a heading, paragraph, or even sentence level can be done relatively
easily with trained narrators, as each of these structures provides a natural
pause point for the person reading, a simplifier not provided when performing
word-level synchronization.</p>
<p data-origPath="/html/body/section/section[1]/p[22]">A real-world recording scenario, for example, would typically involve the
narrator loading their ebook and synchronizing the text in the recording
application as they narrate to speed up this process immensely (e.g., using the
forward arrow or spacebar each time they start a new paragraph to have the
recording program automatically set the new synchronization point). Performing
the synchronization at the natural pause points is not problematic in this
scenario, as the person reading is briefly not focused on that task and/or the
person assisting has enough of a break to cleanly resynchronize. Trying to
narrate and synchronize at the word level, however, is a tricky process to
perform effectively, as people naturally talk more fluidly than any process can
keep up with, even if two people are involved.</p>
</body>
</html>