【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
% a5 V/ G! c& N6 \8 {8 k 但是当时条件不成熟，但是目前来看，条件似乎成熟了

Whisper是openAI的开源语音识别软件。
) r& g2 ~6 O- Q( _% F 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
" ^/ y$ D) N# U& E4 Z4 y! ]
地址如下
https://github.com/sandrohanea/whisper.net
/ R' l, w6 Z- c/ S* d6 ?* T! P* ^$ S
, D/ P1 [! }5 r. u5 [
编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。

编译好之后，有几个注意点
- S1 C( L  q1 V5 w/ M9 \5 ?6 m
: i9 \! ?( G% j. d8 S <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

6 H/ m* V# D  X) t* b/ g <1>Language要设定为"english"。

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   9 L! }1 p1 S$ S6 I
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

- ~8 {5 l) {4 y! V0 o0 b0 b <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
; g# k9 |  x9 C7 o- {) P
. E7 I5 t+ V1 g! T4 y0 c <3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
! `# ?0 X* F5 T2 P8 x  v+ z3 \   （遍历某个目录中的所有文件）
0 r) A  C9 x) m4 C" F
/ R/ V7 M, Q$ Q6 f& D. C <4>输出的文件，需要稍加整理，以符合srt格式
1 g2 ]+ C* D: W* j3 x
   以下是一个Wav文件的控制台输出（幽魂开场动画）

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   ! R/ e# r0 m. t* F$ \0 \
3. whisper_model_load: loading model
   ) H6 I( s; o4 y7 q/ \
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   ' J7 \; f, B3 d/ m4 x
6. whisper_model_load: n_audio_state = 1280
   % z9 D, N) f; c9 ~& t' Y5 L
7. whisper_model_load: n_audio_head  = 20
   : h5 ]3 O# s* L7 ^  o8 k9 h
8. whisper_model_load: n_audio_layer = 32
   7 p8 ^7 @+ i  Z; D# G3 p. H
9. whisper_model_load: n_text_ctx    = 448
   
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    0 v8 |* C7 R9 d4 [9 a( N5 P8 W
13. whisper_model_load: n_mels        = 80
    ; f8 w# B' F) u* Y# n5 w& w
14. whisper_model_load: ftype         = 1
    2 [' f" `1 _8 ?2 j
15. whisper_model_load: qntvr         = 0
    5 k0 q$ p: k0 y
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    + e' @' Q1 B: X: N
18. whisper_model_load: adding 1608 extra tokens
    + P7 \6 x; g- q. `# _
19. whisper_model_load: model ctx     = 2951.27 MB
    
20. whisper_model_load: model size    = 2950.66 MB
    + S% j4 z  K& b7 C1 ^% e% z, d
21. whisper_init_state: kv self size  =   70.00 MB
    1 P- c3 k8 ]' N* l4 @* @" D7 f
22. whisper_init_state: kv cross size =  234.38 MB
    
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    3 V* z3 m- A3 B1 _" z
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    3 I- b+ k' ?9 R1 m3 i3 J$ [6 n3 e- U
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    7 W' B7 ^; X$ |
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    % A6 C3 l9 o7 f7 q! b) }; D
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    + ^3 x6 e1 l9 k5 [/ b
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    4 i& `0 w3 Q7 P0 L% [1 J, M, M* A
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    ( j& O( L9 [' F- V1 M
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    & g9 f( I+ \0 ~- \
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    - A! s% Z* u' k5 ]  ?
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    $ h) Q" p5 p* C- ?$ p
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    ! x2 q; o" @, y" d2 t( s
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    7 D' v0 X  C$ F6 e. I
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    4 O2 ^+ ?! D( j" U* e0 t
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    / G; e! p0 B& n7 l8 b
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    : J5 R' r4 ^5 b7 {0 m. c3 A
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    2 i8 v1 e: X- _( K4 b+ ?
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    8 [. }4 ^! r3 ]1 Y1 B5 @: Q
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    - ^6 l( h6 Q# _6 G, h
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    " j! b! O* [: _/ ~, L$ d
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码

4 U- t6 h6 [# o! }