Also did create a cli. I had to fork the project and removed one unused import which allowed me to remove a lot of unused ml libraries: https://github.com/Mic92/puss-say

I don't think so. It is perhaps a bug to have this unnecessary dependency. I expect the author of kitten to fix this soon.

I wonder if it's possible to guide the intonation in any way.

Nowadays these frustrations shouldn't be a thing any more. If the author used uv, the script would be able to install its own dependencies and just work.

why you don't use some kind of environment, Conda or something like that?

Two words; Nix Flakes

damnn, really sorry for the inconv, looks like some folks are having bad env issues. we're working on fixing this.

I tried it with some "hard mode" text:

The above SECDED check-bit encoding can be implemented in a similar way, but since it uses only three-bit patterns, mapping syndromes to correction masks can be done with three-input AND gates.

It sounded quite good indeed for the normal English stuff, but I guess predictably was quite bad at the domain-specific words. It misspoke "SECDED", had wrong emphasis on "syndromes", and pronounced "AND gates" like "and gates".

Could you give some example of what kind of preprocessing would help in this case? I tried some local LLMs, but they didn't do a good job (maybe my prompts sucked).

He mixed pronounce with enunciate. It's an understandable mistake IMO. (English also has annunciate. Truly a cursed language in many respects.)

https://en.wiktionary.org/wiki/enunciate#English

I have a pipeline of jp epub>m4b, just need to swap tts models in between :)

Exactly. Qwen only has one pitch accent for pure hiragana words, even though it actually work (removing mandarin mixed-in), which requires some great efforts to normalize text in order to disambiguate heteronyms, the result is (if you use voice cloning) your favorite CV speaking in some weird, unknown accent :)

That got me wondering if "you convert to hiragana" is a solved task, or a research team and five years[0], and Google showed me an article[1] that gave me a facepalm, quoting from Google Translate(square brackets are mine):

  > - As a result,
  >   - When the string "明日["tomorrow"]" is entered into TTS, the TTS model [･皿･] outputs an ambiguous pronunciation that sounds like a mix of "asu" and "ashita" (something like "[asyeta]").

  > From this, we found that by using the proposed method, it is possible to obtain data from private data in which the consistency between speech, graphemes, and phonemes is almost certainly maintained for more than 80% of the total.

  > Another possible cause is a mismatch between the domain of the training data's audio (all [in read-aloud tones]) and the inference domain.

  1. Sounds like general state of Japanese speech dataset is a mess
    1.1. they don't maintain great useful correspondence between symbols to audio
    1.2. they tend to contain too much of "transatlantic" voices and less casual speeches
  2. Japanese speakers generally don't denote pronunciations for text
    2.1. therefore web crawls might not contain enough information as to how they're actually pronounced
    2.2. (potentially) there could be some texts that don't map to pronunciations
    2.3. (potentially) maybe Japanese spoken and literal languages are still a bit divergent from each others 
  3. The situation for Chinese/Sinitic languages are likely __nowhere__ near as absurd, and so Chinese STT/TTS might not be well equipped to deal with this mess
  4. This feels like much deeper mess than what commonly observed "a cloud in a sky" Japanese TTS problems such as obvious basic alignment errors(e.g. pronouncing "potatoes" as "tato chi")

  0: https://xkcd.com/1425/
  1: https://zenn.dev/parakeet_tech/articles/2591e71094ea58
  2: https://qiita.com/maishikawa/items/dcadfeebf693080f0415

Hey Rohan, love to contribute feedback.

Huge fan of Ava multilingual and hopefully there are many other others with similar taste, so my feedback might shape things towards a halfway decent direction at least for some.

btw, use case is most often to listen to news/articles.

Bruno was the best of all the voices imo.

Not too bad, actually! Looking forward to hearing some more voices.

No need to DM me, just post on HN or /r/LocalLLama and I'll catch wind of it.

Thanks for your work!

It's mostly torch, I think. It pulls in NVIDIA libs (which … makes sense, I guess), and NVIDIA is just not at all judicious when it comes to disk space. I literally run out of disk trying to install this on Linux.

On macOS, it's a markedly different experience: it's only ~700 MiB there; I'm assuming b/c no NVIDIA libs get pulled in, b/c why would they.

For anyone who might want to play around with this: I can get down to ~3 GiB (& about 1.3 GiB if you wipe your uv cache afterwards) on Linux if I add the following to the end of `pyproject.toml`:

  [tool.uv.sources]
  # This tells uv to use the specific index for torch, torchvision, and torchaudio
  torch = [
      {index = "pytorch-cpu"}
  ]
  torchvision = [
      {index = "pytorch-cpu"}
  ]
  torchaudio = [
      {index = "pytorch-cpu"}
  ]
  
  [[tool.uv.index]]
  name = "pytorch-cpu"
  url = "https://download.pytorch.org/whl/cpu"

This is an example of what one sees under Linux:

  nvidia-nvjitlink-cu12      ------------------------------ 23.83 MiB/37.44 MiB
  nvidia-curand-cu12         ------------------------------ 23.79 MiB/60.67 MiB
  nvidia-cuda-nvrtc-cu12     ------------------------------ 23.87 MiB/83.96 MiB
  nvidia-nvshmem-cu12        ------------------------------ 23.62 MiB/132.66 MiB
  triton                     ------------------------------ 23.82 MiB/179.55 MiB
  nvidia-cufft-cu12          ------------------------------ 23.76 MiB/184.17 MiB
  nvidia-cusolver-cu12       ------------------------------ 23.84 MiB/255.11 MiB
  nvidia-cusparselt-cu12     ------------------------------ 23.99 MiB/273.89 MiB
  nvidia-cusparse-cu12       ------------------------------ 23.96 MiB/274.86 MiB
  nvidia-nccl-cu12           ------------------------------ 23.79 MiB/307.42 MiB
  nvidia-cublas-cu12         ------------------------------ 23.73 MiB/566.81 MiB
  nvidia-cudnn-cu12          ------------------------------ 23.56 MiB/674.02 MiB
  torch                      ------------------------------ 23.75 MiB/873.22 MiB

It also then crashes for me, with:

  File "KittenTTS/.venv/lib/python3.14/site-packages/pydantic/v1/fields.py", line 576, in _set_default_and_type
    raise errors_.ConfigError(f'unable to infer type for attribute "{self.name}"')
  pydantic.v1.errors.ConfigError: unable to infer type for attribute "REGEX"

  -requires-python = ">=3.8"
  +requires-python = ">=3.8,<3.14"

Also:

  + curated-tokenizers==0.0.9

Hilariously, removing it breaks but only on macOS machine. I think you're using it solely for the side-effect that it tweaks phonemizer to use espeakng, but you can just do that tweak yourself, & then I think that dependency can be dropped. That drops a good number of dependencies & really speeds up the installation since we're not compiling a bunch of stuff.

You need to add `phonemizer-fork` to your dependencies. (If you remove misaki, you'll find this missing.)

Emotion based tagging control would be the most helpful narrowing it down. Tags like [sarcastically] [happily] [joyfully] [fearfully]: so a subsection of adverbs.

A stretch goal is 'arbitrary tags' from [singing] [sung to the tune of {x}] [pausing for emphasis] [slowly decreasing speed for emphasis] [emphasizing the object of this sentence] [clapping] [car crash in the distance] [laser's pew pew].

But yeah: instruction/control via [tags] is the deciding feature for me, provided prompt adherence is strong enough.

Also: a thought...

Everyone is using [] for different kinds of tags in this space: which is very simple. Maybe it makes sense to differentiate kinds of tags? I.E. [tags for modifying how text is spoken] vs {tags for creating sounds not specifically speech: not modifying anything... but instead it's own 'sound/word'}

Intonation (frequency rise/fall) would offer a lot of versatility.

not OP but something like [<intention>] where intention might be something like anger, curiousness, etc. [long pause], [gasp], [laughter] stuff like that.

Depends how small or complex you want a TTS, as flite + flitevox voice packages worked on pi or zynq ARM cpu just fine. =3

Also:

https://github.com/sparkaudio/spark-tts

Many of my use cases are similar to those of: Robert J. P. Oberg - (GitHub) ognistik

Perhaps his YouTube channel is worth a watch. This video from four months ago compares various STT tools: https://youtu.be/pKU9CABtnOw

Speaking of apps that would, if I had to guess, love to integrate you:

FluidVoice is incredible and developing quickly. Handy is really hot right now. Also have VoiceInk out there, solid iOS option.

[ps-not parent commenter]

I'm interested in pushing the envelope a bit on the raspberry Pi to do personal assistant projects with it. The pi zero 2 is a surprisingly powerful little device, it is comparable to a pi 3B, except it has less RAM.

From my point of view, Parakeet is not very good at formatting the output, so it would be nice if a small model focused on having nicely formatted (and correct) text, not just the lowest WER score. Rewarding the model for inserting logical line breaks, quotation marks, etc.

Home Assistant integration a la Alexa would be awesome

That's sick, looking forward to it! You have my email in the profile, please let me know when you do!

Listened to the video examples, sounded very good though wasn't terribly challenging text.

If only I could have that in Norwegian my SO would be pleased.

Also I totally misremembered regarding Kokoro TTS. It's good, but not what was butchering Norwegian. Forgot which one I was thinking of, maybe it was the old VITS stuff Rhaspy uses. Points stand, the voice was good but could barely understand what was said.

yes. the current release of our model is english-only. so other languages are not expected to perform well. we'll try to look out for this in our multilingual release.

I want to run it in a website with Wasm and having the browser do the audio playback

desktop CPUs running inference on a single background thread would be the ideal case for what I'm considering.

we'd love to serve this use-case. i'll make a demo for this next week and comment here with it.

I use an epub reader like Moon+ with the built in TTS to turn epubs into audiobooks, and I tried Kokoro TTS but the issue was too much lag between sentences plus it doesn't preprocess the next sentence while it reads out the current one.

What's the source of that voice then in training data? It sounds insanely close to her voice. Strange parallel to openAI denying they trained on Scarlet J's voice.

Right now it's outgoing calls for a small business client that checks information. Although if they call back they don't mind an automated system, on outgoing calls the person answering will often hang up if they detect AI right away, so we use a realistic custom voice with an accent.

This is a mind numbing task that requires workers to make hundreds of calls each day with only minor variations, sometimes navigating phone trees, half the time leaving almost the exact same message.

Anyway, I believe almost all such businesses will be automated within months. Human labour just cannot compete on cost.

You can (just about?) already run on a pi zero, right? That's not literally a chip, but in practical utility it can't be very different

I am actually English but I'm so used to speaking with international people I instinctively say British instead of English - because that's what people expect.

So being factually correct doesn't really matter. Nobody cares and nobody wants to learn so I adapt for them.

In the same way I almost exclusively write with American spelling now. Life is just easier when you stop fighting.

I think you misunderstood the question. I guess its only open-weights not open-source then.

For some insight into the original question, take a look at the Debian ML policy:

https://salsa.debian.org/deeplearning-team/ml-policy

Yeah I have a workflow platform that uses v4.

that would be awesome, I was using pockettts then I had to run it through whisper to get the accurate alignment. Not super productive for realtime work.

So, that was on macOS. It's actually huge on Linux, and I've run out of disk space trying to pull dependencies. It's nvidia, who always shows great judgement in their use of disk.

Spanish would be great, there's a serious lack of Spanish TTS on Android compared to iOS and the quality is not the best.

Here to suggest the Bengali language! It has the 7th largest speaker base worldwide but often ignored by tech companies, sadly.

French, Spanish, German would go a long way.