Jun Sasaki / Log

2026-03-17

Tweaking YouTube Auto-Dubbing

This is an essential step for multi-language expansion, but the accuracy is unfortunately lacking. On the other hand, considering Google's desire to make world languages seamless, I expect it to evolve dramatically over the next year, so I am currently looking for a compromise.
Since major languages like Spanish are stalling, I am currently identifying the issues and testing hypotheses.





Confirmation of conversion to v6 format

  • Conversion from 2-hour Sleep version to 58-minute Cozy Sci-Fi Relax Stories version

  • Meeting the conditions for YouTube auto-dubbing (automatic dubbing in 20 languages) to trigger within 60 minutes

  • 58 minutes is a safety margin for the 60-minute limit

  • Changing positioning from Sleep Stories to BGM for work/reading

  • Changing the number of stories from 7 per video to 4 per video

  • Increasing the number of monthly videos from 8 to 14 (production volume of 56 stories/month remains unchanged)

Improvement of ad structure

  • v5 (Sleep): Ads cannot be inserted into ambient sections → Effectively 1.5 slots/hour

  • v6 (Relax): Can be inserted naturally at breaks in the story → Pre-roll + 2 mid-rolls + end = 4 slots/hour

  • Even though it became shorter, ad density increased by 2.7 times

Auto-dubbing verification

  • Tested Fish voice x 58 minutes x 1 mixed track in Vol. 11 (Volume 009, Chapters 1-4)

  • Confirmed activation of auto-dubbing in 20 languages

  • Standard TTS languages (AR, NL, HE, JA, KO, etc.) were published the same day

  • Expressive Speech target languages (ES, PT, DE, FR, IT, HI, ID) take time to process

  • Expressive Speech is high-quality AI dubbing that includes emotion and intonation

  • All main target languages are covered by Expressive Speech

Technical confirmation from Gemini

  • Processing flow is "Speech detection → Automatic source separation → STT → Translation → TTS → Remix with original BGM"

  • No issues with recognition accuracy at -20dB BGM

  • Not rejected by AI voice (Fish Audio)

  • 30-40 minutes of reading is not judged as "very little speech"

  • A single mixed track is the best practice

  • It is mandatory to set the language to "English" when uploading

Significant cost reduction

  • ElevenLabs $99/month (¥14,850) → $0 by canceling

  • Generate English audio with Fish Audio API. About ¥515 for 28 episodes in 1 volume

  • About ¥1,030 for 2 volumes per month

  • Suno ¥1,500 (BGM)

  • Monthly total ¥16,350 → ¥2,530 (85% reduction)

▶︎I had an annual contract for Suno, haha. It's a negligible cost of ¥1,200, so it's fine.

rt2112_voice.py v6 complete

  • Direct HTTP calls instead of Fish Audio SDK (Python 3.9 support)

  • Built-in text formatting: removal of ◇, ———, ## symbols, and URL removal

  • Manuscript files are not changed at all (read-only)

  • Cost can be checked with dry-run

  • Setup is just pip install requests and setting FISH_API_KEY

▶︎To generate voice directly from the Kindle manuscript, I programmatically remove elements that could cause howling before passing it through the API. Honestly, the accuracy isn't great as the voice sometimes changes in the middle of an 8-minute reading, but I'm compromising on that for now. ElevenLabs is too expensive and doesn't sound that different for the price, so Fish is the only choice for now. Since it can't export wav files, I'll definitely reconsider after monetization. Either way, my policy is that I don't require that much voice accuracy for my content.



BGM dB test

  • Standard language auto-dubbing passes at both -12dB and -15dB

  • The Expressive Speech stack issue is caused by YouTube's processing queue/bugs, not BGM volume

  • The balance after auto-dubbing is more natural at -15dB (at -12dB, the dubbed voice becomes too strong)

  • Final decision: Decide BGM volume by feel to match the song. Prioritize the listening experience of the EN version

▶︎At first, I thought BGM was interfering with auto-dubbing, but that's not really the case. Honestly, you don't need to worry about it. Considering translation accuracy, it's more for background music while working, so I want to aim for the limit with the volume.

Loudness setting adjustment

  • Premiere's Essential Sound Auto-Match was -23 LUFS (broadcast standard)

  • YouTube's recommendation is -14 LUFS

  • Turn off loudness in Essential Sound

  • Set loudness normalization to -14 LUFS in the Effects tab when exporting

  • Adjust BGM dB by ear in Premiere, and let overall volume be auto-adjusted during export

▶︎Gemini is full of it. "Premiere has a feature that can do it in one go!" → It's not for YouTube specs. In the end, adjusting it yourself is best.


Breakthrough in auto-dubbing discovered

  • You can upload subtitle text manually from the Language tab in YouTube Studio

  • Auto-generate subtitles for each language from English subtitles using the Auto Translate feature

  • Delete stuck auto-dubbing → Re-add subtitles via Auto Translate → Potential for reprocessing

  • Subtitles confirmed as Published in French, German, and Indonesian

  • Verifying if audio tracks will be generated

▶︎Expressive Speech languages are all stuck. It seems there are cases where they were published by re-uploading subtitles. So, I will re-upload subtitles for each language and see how it goes.

Provisional schedule for posting operations

  • 3-4 videos per week (pace of 14 per month)

  • Post at 23:00 JST (= 12:00 PM NY time)

  • Set scheduled posts to go live 3 days after upload (waiting for auto-dubbing to complete)

▶︎The algorithm is expected to start working once I reach 30 videos, so I'll be boosting this month. Aiming to reach 30 videos as fast as possible. Starting the above operation next month.
Currently, work is completed within 1 hour per video.

Channel data (as of March 17)

  • Number of published videos: approx. 16

  • CTR for Vol. 11: 4.1% (achieved target of over 4%)

  • Unique viewers are still low, but recommendations have started

  • Vol. 15 appeared in recommendations as "5 more than usual" immediately after publication

Production flow finalized (v6)

  • Script: md as is

  • Audio: rt2112_voice.py → Fish API → mp3

  • Editing: Proofreading + BGM adjustment in Premiere

  • Video: Gemini/Veo + Premiere (multiple variations using 8-second footage playback and reverse playback patterns)

  • Export: -14 LUFS normalization

  • Upload: YouTube (language setting English) → Scheduled for 23:00 JST

  • Multilingual: Auto-dubbing automatic + supported by Auto Translate subtitles

Goals

  • 500,000 yen/month (must-achieve): 670,000 monthly views, within 1 year

  • 1 million yen/month (stretch): 1.33 million monthly views

  • Monetization (YPP): Expected to reach with 30-50 videos

▶︎ Both Claude and Gemini say that, but I feel like monetization is probably impossible. Let's just keep going calmly.

Next steps

  • Check the results of auto-dubbing regeneration from Auto Translate subtitles

  • Reach 30 videos by the end of this month

  • Turning Kindle volume 8 into a video (even though I'm currently turning volume 9 into a video for some reason)

Advertisement