【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
' D( d1 p+ s; c% T. A4 U 但是当时条件不成熟，但是目前来看，条件似乎成熟了
" J" M" V5 @8 j
Whisper是openAI的开源语音识别软件。
它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
; n5 f5 D; _3 w3 d4 G 之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
" F9 b0 W8 ]( I- I( ]/ u
; C8 ?8 E/ z9 v% \ 地址如下
  v  V( t0 }5 ]/ g7 x( H https://github.com/sandrohanea/whisper.net
4 k5 v8 N* U7 L9 y/ J
6 t+ [/ G5 u3 L8 E; W. Y
! j( U/ J3 e9 s2 L  Z5 v 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。
" ^' O! z% v4 U5 u3 K$ f  U
编译好之后，有几个注意点
% X/ \) p& Z3 d$ o" I7 |6 e
<0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。
  E6 L; u/ J: F( p) D
. ~9 A, u: c0 l; s# F <1>Language要设定为"english"。

1. /*    var builder = factory.CreateBuilder()
   % n8 o1 M( E' m. z" _0 e, n
2.         .WithLanguage(opt.Language);*/
   
3.     var builder = factory.CreateBuilder()
   4 R8 z- _& [+ V1 b1 {  D6 Z
4.     .WithLanguage("english");

复制代码

" E+ G. N4 d- C# `! u, b) h% q4 A/ a <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
! E5 T) I8 \- x; ^2 l6 ~
<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
   （遍历某个目录中的所有文件）
5 P1 h3 x0 w" y+ x
0 I( J$ i1 k5 \/ w* }+ `1 T <4>输出的文件，需要稍加整理，以符合srt格式

, L" k4 |5 B& N( H9 b8 K   以下是一个Wav文件的控制台输出（幽魂开场动画）

1. 
   1 n- Q8 Y) X) _0 r
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   ) _+ u+ Z) A  Y5 F' O( y
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   & D1 }3 u3 H+ ]& U, R+ b
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   9 y8 |6 V6 |# o3 o  M- a( `
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   - t$ y5 b. p* _1 i. q: z# F
9. whisper_model_load: n_text_ctx    = 448
   
10. whisper_model_load: n_text_state  = 1280
    6 ~5 p: U5 }8 {1 C1 k
11. whisper_model_load: n_text_head   = 20
    / B; {& t' g" X" v+ o5 [, ]
12. whisper_model_load: n_text_layer  = 32
    3 q" M% x# O. N) c$ c* y4 O
13. whisper_model_load: n_mels        = 80
    . Q+ U# r6 O  `6 I8 c3 A- b/ X- |
14. whisper_model_load: ftype         = 1
    
15. whisper_model_load: qntvr         = 0
    
16. whisper_model_load: type          = 5
    5 I7 @' h. |4 W. j$ @
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    " {# X( B: a; k* J
18. whisper_model_load: adding 1608 extra tokens
    + z- [# r/ }# {( t, c% c
19. whisper_model_load: model ctx     = 2951.27 MB
    + c: L# O, P! W4 v0 u# g
20. whisper_model_load: model size    = 2950.66 MB
    
21. whisper_init_state: kv self size  =   70.00 MB
    
22. whisper_init_state: kv cross size =  234.38 MB
    3 _) V8 V2 \1 X' z, M$ I
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    7 |: N3 ]& b, H0 f$ ?
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    3 X. c8 S( a6 R6 H  _$ i
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    8 {" U% n+ p: u/ \9 d
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    : s5 Z, ~8 j9 H  l% c$ O6 |4 I) O
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    # Y/ p6 y. V1 B8 A+ A! }8 d
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    - L1 g  q, {2 t! S  O
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    ) m7 N6 D/ O; `! }- f  ^+ r
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    4 Q" M( B0 _$ r3 v  \: s0 B: O
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    ' s. n4 Q- F' N6 C! g5 U1 c3 m
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    ; x8 }" z) o  M
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    , C( t7 Q  Q" H% {
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    / m" o+ _- H: O0 y( `, r
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    / u4 Q" o: U' x5 {

复制代码

- ?7 S7 g* T5 P2 N, S( \& S* `