【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
但是当时条件不成熟，但是目前来看，条件似乎成熟了

' J" `. {' _: F: \& ^3 O Whisper是openAI的开源语音识别软件。
( Z! {9 y  Q/ } 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
* a) S6 w# \  d; _4 b! Y/ E6 Z* l
% j# V# Y& x# Q; G/ R  [ 地址如下
https://github.com/sandrohanea/whisper.net
( C* P2 J: l! F1 w# Y
# f# }9 n: a) a  k( H# a
. F$ k0 e: z6 L 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。
' s% ]$ s6 H% l, n
3 F6 F% t6 O& d6 `7 r8 F# @, v0 D$ n 编译好之后，有几个注意点
: k2 B! c1 W& T
' @: P& B+ v" |( q6 q' m. ^ <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
  m1 E! c+ c% K) N* n    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

<1>Language要设定为"english"。
. R2 B1 ?6 l: c: I0 n0 X

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

/ X8 ^. F. ]1 g <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
. v* v. @: E  }3 w+ g- D0 Q: }; `
<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
, S4 A' t$ u& G' {% X. @   （遍历某个目录中的所有文件）

* [; U0 A) `4 Q% d <4>输出的文件，需要稍加整理，以符合srt格式

   以下是一个Wav文件的控制台输出（幽魂开场动画）
& {' \% c% }( V" }( C4 j6 c8 z

1. 
   " F1 N7 h6 u$ {: U, p, `
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   7 k" G9 U8 x: G4 x
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   : t5 X& h. }/ [. W* n
6. whisper_model_load: n_audio_state = 1280
   8 b4 D5 k8 q2 |# e$ B: d( V
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   
9. whisper_model_load: n_text_ctx    = 448
     T) P- j" H' J# r1 w( N
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    
13. whisper_model_load: n_mels        = 80
    
14. whisper_model_load: ftype         = 1
    
15. whisper_model_load: qntvr         = 0
    % v5 a; a& D  w$ _/ c& C2 m
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    
19. whisper_model_load: model ctx     = 2951.27 MB
    . X5 A/ X6 p0 T. T8 N3 k4 g
20. whisper_model_load: model size    = 2950.66 MB
    , q- i5 q/ f/ I
21. whisper_init_state: kv self size  =   70.00 MB
    
22. whisper_init_state: kv cross size =  234.38 MB
    9 J. P8 M& v1 V/ m' G
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    ; p2 P& t0 c5 k0 x
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    2 R  i" L5 F8 p" F9 j" N) F9 X
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    1 y# \2 h! V; d7 a4 p8 {, Z
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    + S8 e. J6 l+ n3 x) l+ g
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    ' d7 q9 h. a' `! a9 s2 }
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    : w* I' i2 M3 n, Y
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    ! X% R0 E  Q$ F9 e% s- P
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    . `; S1 J  K8 Y$ V3 U% V
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    6 M: B# N% V3 p5 ~6 M+ s
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    5 t4 D* \/ I# ^
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    5 k8 W, s# R. r, P, I
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    ( N# |7 G3 z3 i& t: \
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    ' f0 N: Y: x; T
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码