Hello,

I am running the latest version of Gentle on a Mac. In server mode, it runs through the first pass fine, but when it runs the second pass (the realignment phase), it gets to about half way and then it stops processing. There are no errors, but the k3 background processes all go away, except for one, and CPU usage drops to 0%. I think it might be a problem with the multipass.realign() function, standard_kaldi.py, or even the k3.cc file itself.

This seems to only occur on longer audio files.

Thank you @strob

EDIT: I did some more testing, and align.py also will not finish if the audio file is too long. (Do note the test files work fine, it's just longer files I have problems with. In addition, the 2017 version of Gentle did not have this issue)

I am having an issue installing from source on Ubuntu 18.04.4 LTS. While installing Kaldi, the makefile doesn't get created. I have tried all suggested fixes in #192, as well as Kaldi issue 3395 (https://github.com/kaldi-asr/kaldi/issues/3395).

Since upgrading to El Capitan I haven't been able to install the included version of Kaldi. There's an issue with gcc-fortran. Were you able to get it compiling?

Hello, thanks for gentle!

I'm trying to install on centos 7, installed dependencies with yum install, here is my install.sh:

#!/bin/bash

set -e
git submodule init
git submodule update

#./install_deps.sh
pip install .
(cd ext && ./install_kaldi.sh)
./install_models.sh
./install_language_model.sh
cd ext && make depend -j 2 && make -j 2

But somewhre in the make process this happens (after ./install_language_model.sh is executed):

make[1]: Leaving directory `/root/src/gentle/ext/kaldi/src/util'
g++ -rdynamic   k3.o kaldi/src/online2/kaldi-online2.a kaldi/src/ivector/kaldi-ivector.a kaldi/src/nnet3/kaldi-nnet3.a kaldi/src/chain/kaldi-chain.a kaldi/src/nnet2/kaldi-nnet2.a kaldi/src/lat/kaldi-lat.a kaldi/src/decoder/kaldi-decoder.a kaldi/src/cudamatrix/kaldi-cudamatrix.a kaldi/src/feat/kaldi-feat.a kaldi/src/transform/kaldi-transform.a kaldi/src/gmm/kaldi-gmm.a kaldi/src/hmm/kaldi-hmm.a kaldi/src/tree/kaldi-tree.a kaldi/src/matrix/kaldi-matrix.a kaldi/src/fstext/kaldi-fstext.a kaldi/src/util/kaldi-util.a kaldi/src/base/kaldi-base.a   /root/src/gentle/ext/kaldi/tools/openfst/lib/libfst.a /root/src/gentle/ext/kaldi/tools/OpenBLAS/install/lib/libopenblas.a -lgfortran -lm -lpthread -ldl  -o k3
m3.cc: In function 'int main(int, char**)':
m3.cc:174:28: error: no matching function for call to 'AddSelfLoops(kaldi::TransitionModel&, std::vector<int>&, float&, bool, bool, fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*)'
                   &hclg_fst);
                            ^
m3.cc:174:28: note: candidates are:
In file included from kaldi/src/fstext/pre-determinize.h:94:0,
                 from kaldi/src/fstext/fstext-utils-inl.h:29,
                 from kaldi/src/fstext/fstext-utils.h:448,
                 from kaldi/src/fstext/context-fst-inl.h:23,
                 from kaldi/src/fstext/context-fst.h:535,
                 from m3.cc:1:

and later on:

make: *** [m3.o] Error 1
make: *** Waiting for unfinished jobs....
kaldi/src/ivector/kaldi-ivector.a(ivector-extractor.o): In function `kaldi::IvectorExtractor::ComputeDerivedVars()':
/root/src/gentle/ext/kaldi/src/ivector/ivector-extractor.cc:198: undefined reference to `kaldi::g_num_threads'
kaldi/src/ivector/kaldi-ivector.a(ivector-extractor.o): In function `kaldi::IvectorExtractorStats::IvectorExtractorStats(kaldi::IvectorExtractor const&, kaldi::IvectorExtractorStatsOptions const&)':
/root/src/gentle/ext/kaldi/src/ivector/ivector-extractor.cc:807: undefined reference to `kaldi::Mutex::Mutex()'

Any advice? (by the way, the the latest cuda (9.2 as of writing this) is not supported by gentle's kaldi i guess)

Thanks a lot.

Hey, I'm trying to run a gentle command. But when I do, I get this error.

Can anyone pls explain what I should do?

(simple explanations cus im a total noob)

INFO:root:converting audio to 8K sampled wav
INFO:root:starting alignment
Traceback (most recent call last):
  File "align.py", line 54, in <module>
    aligner = gentle.ForcedAligner(resources, transcript, nthreads=args.nthreads, disfluency=args.disfluency, conservative=args.conservative, disfluencies=disfluencies)
  File "gentle/forced_aligner.py", line 18, in __init__
    gen_hclg_filename = language_model.make_bigram_language_model(ks, resources.proto_langdir, **kwargs)
  File "gentle/language_model.py", line 126, in make_bigram_language_model
    raise e
  File "gentle/language_model.py", line 116, in make_bigram_language_model
    subprocess.check_output([MKGRAPH_PATH,
  File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ext/m3'

any help is appreciated. thx!

i installed this aligner on linux which at last shows this:

Fetched 5,296 kB in 1min 6s (80.2 kB/s)                                        
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package ffmpeg is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'ffmpeg' has no installation candidate

after that when i execute python serve.py it gives this error:

Traceback (most recent call last):
  File "serve.py", line 23, in <module>
    from gentle import multipass
  File "/home/ucertify/Desktop/gentle/gentle/multipass.py", line 15, in <module>
    with open(vocab_path) as f:
IOError: [Errno 2] No such file or directory: 'PROTO_LANGDIR/graphdir/words.txt'

Please help me...

I do the following, but the installation is going forever:

!git clone -q https://github.com/lowerquality/gentle.git > /dev/null
%cd /content/gentle
!./install.sh

When I run install.sh on Linux, it eventually stops at this error.

Makefile:5: kaldi/src/kaldi.mk: No such file or directory Not building with cuda!!! make: *** No rule to make target 'kaldi/src/kaldi.mk'. Stop.

the ext/kaldi/src direct has a bunch of other things but is indeed missing kaldi.mk file.

Hi, thanks for making Gentle available!

I'm looking into using Gentle, but without running a server. This PR includes:

• align.py, a shell command that runs the forced alignment algorithm and outputs JSON
• To do that cleanly, I factored out the algorithm from serve.py, creating two classes ForcedAligner and FullTranscriber
• And to do that cleanly, I bundled all resource lookups into a single GentleResources instance which I pass around; and once I had that I propagated it into the other bits of code that used those resources.

I took my best shot at organizing and naming and whatnot, but of course I'm open to changes that'd you'd find more suitable.

[Oh I should also point out that I haven't tested FullTranscriber since the install didn't seem to include data/graph/HCLG.fst. It "should" work since the code is just moved over from where it had been earlier, but of course bugs could have easily crept in.]

gentle0.03.dmg does not start; may be related to spurious homebrew packages on my system.

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   org.python.python               0x000000010324d060 PyTuple_New + 112
1   org.python.python               0x000000010325049e PyType_Ready + 199
2   org.python.python               0x0000000103250460 PyType_Ready + 137
3   org.python.python               0x000000010323d19a _Py_ReadyTypes + 16
4   org.python.python               0x00000001032a6475 Py_InitializeEx + 395
5   org.pythonmac.unspecified.gentle    0x00000001000024ac 0x100000000 + 9388
6   org.pythonmac.unspecified.gentle    0x000000010000117a main + 650
7   org.pythonmac.unspecified.gentle    0x0000000100000be4 start + 52

I'm trying to use Gentle to create a lip-flap animator, and while the software works for me, I don't really know how to use its outputs.

In my program, each phoneme of the English language will be assigned to a photo, and so I need to know what each phoneme uttered is. Gentle gives me this data, but the phoneme names don't seem to match any digitization of, say, the IPA. Instead, I get obscure phoneme names like dh_B and ah_E. dh is usually transcribed as th so this caught me off-guard at first, and I have no idea what E means.

If I could get a list of all of Gentle's possible phonemes and their mappings to the IPA, that would make my life a lot easier.

I am getting this error when trying to call align.py from the command line on Mac OS 10.12.6 Sierra, Python 3.11:

$ python3 align.py /Volumes/.../5077_FINAL.WAV /Volumes/.../RAY15077_cleaned.txt Traceback (most recent call last): File "/Users/.../gentle/align.py", line 49, in resources = gentle.Resources() ^^^^^^^^^^^^^^^^^^ File "/Users/.../gentle/gentle/resources.py", line 19, in init require_dir(self.proto_langdir) File "/Users/.../gentle/gentle/resources.py", line 16, in require_dir raise RuntimeError("No resource directory %s. Check %s environment variable?" % (path, ENV_VAR)) RuntimeError: No resource directory /Users/.../gentle/exp. Check GENTLE_RESOURCES_ROOT environment variable?

It looks like the issue is that the gentle.Resources() method is looking for a path that ends in "gentle/exp", but that folder isn't in the directory structure.

Does anyone have any suggestions to run this from the command line on a Mac? I don't currently have access to Docker.

I'm not a developer but I do find Gentle very useful.

Since OpenAI released their Whisper models last week, I've been wondering if anyone with development skills would be interested in enabling an option to utilize Whisper instead of Kaldi when running Gentle.

I know that language support for spoken languages beyond English has been a long-standing request for Gentle. Whisper appears to be pointedly multi-lingual, so perhaps this would make support for languages beyond English more easily achievable for Gentle?

Anyway, please let me know what scale of an undertaking this would be. Thanks in advance.

The prebuilt Mac application will not open on MacOS 12.5. I have option clicked selected open and it simply flickers and immediately closes.

Model Name: MacBook Pro Model Identifier: MacBookPro18,2 Chip: Apple M1 Max Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 64 GB

Hi I am trying to find out if the phonemes that are listed in the output are the actual phone utterances or just listed from the dictionary. I am trying to find out if phoneme was correctly pronounced? Is there a way to identify the mispronounced phonemes in Gentle output

I trained a kaldi tdnn model and and modified it for gentle format I tried running align.py by it causes the following error ! Can anyone help me with this?

/var/folders/bn/gms2j96s50x1ts27vgwp1jfm0000gp/T/tmpv3dkq1cp_HCLG.fst ERROR:gentle.standard_kaldi:hclg_path does not exist: /var/folders/bn/gms2j96s50x1ts27vgwp1jfm0000gp/T/tmpv3dkq1cp_HCLG.fst /Users/adminuser/Documents/gentle/gentle/exp/tdnn_7b_chain_online/ /var/folders/bn/gms2j96s50x1ts27vgwp1jfm0000gp/T/tmpv3dkq1cp_HCLG.fst ERROR:gentle.standard_kaldi:hclg_path does not exist: /var/folders/bn/gms2j96s50x1ts27vgwp1jfm0000gp/T/tmpv3dkq1cp_HCLG.fst Traceback (most recent call last): File "/Users/adminuser/Documents/gentle/gentle/align.py", line 58, in result = aligner.transcribe(wavfile, progress_cb=on_progress, logging=logging) File "/Users/adminuser/Documents/gentle/gentle/gentle/forced_aligner.py", line 24, in transcribe words, duration = self.mtt.transcribe(wavfile, progress_cb=progress_cb) File "/Users/adminuser/Documents/gentle/gentle/gentle/transcriber.py", line 51, in transcribe pool.map(transcribe_chunk, range(n_chunks)) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 364, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 771, in get raise self._value File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar return list(map(*args)) File "/Users/adminuser/Documents/gentle/gentle/gentle/transcriber.py", line 38, in transcribe_chunk k.push_chunk(buf) File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 39, in push_chunk self._p.stdin.write(buf) #arr.tostring()) BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <function Kaldi.del at 0x7fcdccd76c10> Traceback (most recent call last): File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 79, in del File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 73, in stop File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 30, in _cmd BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <function Kaldi.del at 0x7fcdccd76c10> Traceback (most recent call last): File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 79, in del File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 73, in stop File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 30, in _cmd BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <function Kaldi.del at 0x7fcdccd76c10> Traceback (most recent call last): File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 79, in del File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 73, in stop File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 30, in _cmd BrokenPipeError: [Errno 32] Broken pipe Exception ignored in: <function Kaldi.del at 0x7fcdccd76c10> Traceback (most recent call last): File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 79, in del File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 73, in stop File "/Users/adminuser/Documents/gentle/gentle/gentle/standard_kaldi.py", line 30, in _cmd BrokenPipeError: [Errno 32] Broken pipe

Initially I got a called subprocess error I tried to decode the output by changing line 116 in language_model.py to

try: devnull = open(os.devnull, 'wb')

    print([MKGRAPH_PATH,
                    proto_langdir,
                    txt_fst_file.name,
                    hclg_filename])

    sub_result = subprocess.check_output([MKGRAPH_PATH,
                    proto_langdir,
                    txt_fst_file.name,
                    hclg_filename],
                    stderr=devnull)
except subprocess.CalledProcessError as e:
    sub_result = e.output.decode()