【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
7 v4 p& D6 p+ E$ y* z 但是当时条件不成熟，但是目前来看，条件似乎成熟了
! l" P6 C; }& H
+ B$ @+ [9 H- m3 Q Whisper是openAI的开源语音识别软件。
; F9 L1 C6 G2 H( R) C# N 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
2 E2 y  I0 H+ i0 G 之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
, o6 x& p3 j% s' I
地址如下
9 g% U! i3 W2 \  I9 P+ j# w https://github.com/sandrohanea/whisper.net
) G5 w4 @; C" W3 ]( `, P/ W

% M, M. h5 D4 \& E2 {, _ 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。
% A  T/ C5 q8 Z3 ^" [  C6 I
( u! Y: y' t5 o' O9 F) i 编译好之后，有几个注意点
4 P) M* z( j) ^6 B$ a& u: j
<0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。
1 w: r4 S. X9 N
1 k/ n, ]3 t# h6 Y+ {1 d <1>Language要设定为"english"。
% Z- X; [# _& Q8 `% u* f8 ?

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   
3.     var builder = factory.CreateBuilder()
   ; y3 {6 t! p3 `3 H" J
4.     .WithLanguage("english");

复制代码

<2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
' x2 n; x; B* ]+ C0 R8 Y+ `5 k+ Y/ E
# }7 I: e' _& k1 V( F! A <3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
   （遍历某个目录中的所有文件）
. m$ w/ ]2 |) O: I3 s% z
<4>输出的文件，需要稍加整理，以符合srt格式

9 H- G7 ~3 H  w# a   以下是一个Wav文件的控制台输出（幽魂开场动画）

1. 
   7 S# k& x9 {" p' S
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   - z+ O  g  e7 r, L7 A
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   
9. whisper_model_load: n_text_ctx    = 448
     \( q7 u. p# ?; u2 Q# ?
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    5 A8 R+ G2 y- F1 M0 f1 K2 S
13. whisper_model_load: n_mels        = 80
    
14. whisper_model_load: ftype         = 1
    4 T& m  j3 A& s3 `
15. whisper_model_load: qntvr         = 0
    . ?8 a$ S$ |  n! b' E' \1 n; E) E
16. whisper_model_load: type          = 5
    $ f- p2 l: @7 W4 I+ x
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    & T) ]5 Q3 M. |: m2 o
19. whisper_model_load: model ctx     = 2951.27 MB
    
20. whisper_model_load: model size    = 2950.66 MB
    # }: ?1 r% z) g: u& T: u
21. whisper_init_state: kv self size  =   70.00 MB
    * y- z' `1 e7 I% d
22. whisper_init_state: kv cross size =  234.38 MB
    6 _. j8 d" m* ?6 \
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    5 A; Q7 C; S2 |1 W0 a6 e
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    % t. N1 G7 x- L( m* Q& |6 Y8 h
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    # P% ^! a3 U( S4 s+ N. t& |2 c: }
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    . \9 A+ r3 G$ O% Z
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    1 W9 J$ {& r- M, n9 T  {
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    $ |! d3 q0 J0 R3 D0 g
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    0 m. I6 G3 A- }- G
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    + z0 W, s$ b* |- A
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    ' Y# k% F, U$ f. o4 |
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    # m  c7 `( t: T. j
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    / k7 m+ e! B+ V7 {: J& ^' ^
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    & O6 o& u# R, t5 F- O
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    , L  P. {2 i$ r" _/ X3 e2 R
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    8 a" r: k7 r8 K4 }& k! |% ?
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    , R. Y$ |7 \0 B9 g$ x9 t; f! Q
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    - f6 N+ `- t3 N3 [
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    # |; F5 s6 s- p
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    1 x# F2 u4 {/ b9 H6 H
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    $ }  }6 Z/ L' T5 N1 y$ |
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码

1 g9 t5 S" `& w& ]0 t9 G