【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
但是当时条件不成熟，但是目前来看，条件似乎成熟了

Whisper是openAI的开源语音识别软件。
它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
0 ]1 H+ u. I# p4 P
" F+ X! A2 r; c1 C8 ]6 G2 { 地址如下
; P) b. Q) J* }. | https://github.com/sandrohanea/whisper.net

% `8 t' D$ a) S" S7 O& F
$ A" R7 _, |1 F4 O 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。
9 ~/ J) G9 Y1 g9 [0 f
编译好之后，有几个注意点

& m1 B6 f8 g/ n5 {( o- p3 T <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
; B* d& j: b8 z  q' I! a1 z3 e    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

<1>Language要设定为"english"。
8 l2 ^8 v2 a, G
9 T. F, _4 ^% j! r! ?

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   5 h0 \) w7 R% v4 ~
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

8 S/ ~1 p8 k6 a% S/ }! F/ ^. J <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
4 Z9 ?4 ~8 G" L9 I5 j
<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
   （遍历某个目录中的所有文件）
% d3 z. r, ~3 J; s9 E; j) V
<4>输出的文件，需要稍加整理，以符合srt格式

   以下是一个Wav文件的控制台输出（幽魂开场动画）

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   ' Y% b& d" p5 Y. ?, C" ]( v
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   
7. whisper_model_load: n_audio_head  = 20
   , i1 A& k  Q8 Q  R# @3 h
8. whisper_model_load: n_audio_layer = 32
     @$ h; d$ v7 @9 ?& R: k4 O* S
9. whisper_model_load: n_text_ctx    = 448
   
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    
13. whisper_model_load: n_mels        = 80
    
14. whisper_model_load: ftype         = 1
    * F, W6 {1 u2 L- o$ V% N, v" A. ~
15. whisper_model_load: qntvr         = 0
    
16. whisper_model_load: type          = 5
    + Z0 G! W- Z  ]" m/ Q: \
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    : Z+ U6 o6 a+ f
18. whisper_model_load: adding 1608 extra tokens
    + p* A  Z7 @+ q; y1 Z9 M! E
19. whisper_model_load: model ctx     = 2951.27 MB
    0 u/ a! x- W+ O9 @
20. whisper_model_load: model size    = 2950.66 MB
    
21. whisper_init_state: kv self size  =   70.00 MB
    . L2 T  C2 ?# A. K% V
22. whisper_init_state: kv cross size =  234.38 MB
    5 h8 a  y  F  p/ t
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    : o( _% n6 Y8 j" M) p4 x
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    9 N+ D1 N) i, y& @* o6 C
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    $ s/ e: h$ E' {( [
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    1 N1 \( ]" x- }$ z
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    4 A5 E# P' \! o0 r  J9 y$ q
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    6 d6 ]. W, m, ?  s9 n
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    ' f" h: ^# b$ o4 M2 c
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    / B6 h( h( ]9 I9 O$ v
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    " V/ n) A2 t  S! |5 D
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    ; `! d. S+ o1 O) J
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    ' h, d" m, g0 \
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    # {3 J( q' A: a% `
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    ' C  Q% y- z6 [2 X
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
      V3 u- I9 j# U7 k: G' Y8 y8 c/ }, J9 @
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码