Semi-related: Wikipedia's homepage also contains a very minimalist, manually curated news section with only major world events:
https://en.wikipedia.org/wiki/Portal:Current_events
Although that's perhaps way too minimalist?
Wikipedia's current events selection has little to do with significance, being overly focused on events that cause multiple deaths and political events, regardless of their global significance.
Not trying to have an edgy take here, it comes from a genuine attempt to use it for this exact purpose, but it didn't work at all. A train crash killing 20, as unfortunate as it is, can't possibly be one of the 5 most significant things that happened over the last week, but Wikipedia current events considers it one.
Wikipedia current events page was actually one of the reasons for creating this project!
I was disagreeing a lot with their selection of news, for example one of their recent entries is:
"Two people are killed and eleven others are injured when a bus flips on its side on a highway near Prenzlau, northeast of Berlin, Germany."
My system gave it a significance score of 1.8, so similar news should never get to the main page: https://www.newsminimalist.com/articles/two-dead-and-four-in...
I hadn't seen this comment and just wrote pretty much a mirror one - Wikipedia overly focuses on one-off accidents and events, rather than news with long-term implications.
Does your service do a good job at thinking longer term? Would you have an example of this?
It's hard to measure, but I see such stories sometimes, mostly in technology and science.
A couple recent examples:
[Significance 6.7] China outlines ambitious space strategy to surpass US by 2050 [1]
[Significance 5.9] Scientists explore ocean floor findings that could reveal oxygen on other planets [2]
[1] https://www.newsminimalist.com/articles/china-outlines-ambit...
[2] https://www.newsminimalist.com/articles/scientists-explore-o...
Isn't significance heavily subjective though?
A lot of the most signficant stories are political, for example, which someone may have no interest in.
I have had this same idea in the past, tuning to my personal interests.
Good point! I actually have this exact question on about page [0], I'll copy my thoughts from there:
I separate significance from importance (or relevance).
Importance is subjective. News about the health of my family members is important to me, but it is not significant to the world.
Significance is objective. It's about how much the event affects humanity as a whole.
> Significance is objective. It's about how much the event affects humanity as a whole.
I don’t agree with that, at least not in the present. We only know what’s truly significant when we reflect on history. There are very few things we can be certain are significant in the present. Climate change is likely one, but the US debt ceiling and the war in Ukraine don’t seem as likely to me, at least not in the human scale. There are also events that happen that don’t appear significant in the present but will be hugely significant in the future.
I actually agree that "true" significance can only be estimated in hindsight.
My goal with this project is not to get "true" significance but to have a setup that gets you 90% of the way there: an automated system that finds events that are likely to affect large groups of people or major systems and filters out most of everyday noise.
There will always be false-positives and false-negatives, but I think it's a good starting point and it should slowly get better as models get smarter.
Significant Others just got downgraded to Important Others
Makes sense that way. I think you nailed the execution. Good job!
What’s the cost of running something like this?
Wow, I really have wanted something like this. Have you spent a bit of work on prompting gpt-4 to get good results? Am curious what you found worked the best for your app
I've been using rss feed for few months but recently it became borderline useless. For example here is grep of pubDate:
$ wget -qO - https://rss.beehiiv.com/feeds/4aF2pGVAEN.xml | grep pubDate
<pubDate>Thu, 16 Jan 2025 17:56:47 +0000</pubDate>
<pubDate>Thu, 16 Jan 2025 17:56:47 +0000</pubDate>
<pubDate>Mon, 13 Jan 2025 17:22:34 +0000</pubDate>
<pubDate>Sat, 11 Jan 2025 16:42:01 +0000</pubDate>
<pubDate>Thu, 09 Jan 2025 17:52:38 +0000</pubDate>
<pubDate>Fri, 03 Jan 2025 16:19:25 +0000</pubDate>
<pubDate>Sat, 28 Dec 2024 17:44:35 +0000</pubDate>
<pubDate>Wed, 18 Dec 2024 16:44:00 +0000</pubDate>
<pubDate>Mon, 16 Dec 2024 18:01:56 +0000</pubDate>
<pubDate>Tue, 10 Dec 2024 18:00:55 +0000</pubDate>
<pubDate>Fri, 06 Dec 2024 17:43:37 +0000</pubDate>
<pubDate>Wed, 04 Dec 2024 16:38:38 +0000</pubDate>
<pubDate>Sun, 01 Dec 2024 18:57:43 +0000</pubDate>
<pubDate>Fri, 29 Nov 2024 18:07:14 +0000</pubDate>
<pubDate>Sun, 24 Nov 2024 15:26:59 +0000</pubDate>
<pubDate>Thu, 21 Nov 2024 17:25:22 +0000</pubDate>
<pubDate>Tue, 19 Nov 2024 17:47:19 +0000</pubDate>
<pubDate>Fri, 15 Nov 2024 17:28:00 +0000</pubDate>
<pubDate>Tue, 12 Nov 2024 17:44:08 +0000</pubDate>
<pubDate>Fri, 08 Nov 2024 17:22:01 +0000</pubDate>
<pubDate>Wed, 06 Nov 2024 16:43:10 +0000</pubDate>
This rss is not exactly an rss with articles from the main page, but a newsletter I send manually every few days once enough significant stories happen to warrant an email. Each newsletter issue includes 2-5 main articles and 3-7 trending articles.
As an RSS user, I would love an RSS of the main page content, one entry per story over 5.5 is a perfectly reasonable baseline.
Also: It'd be great if you had a feed tag in your HTML head, so RSS readers could pick it up straight out of your homepage URL instead of needing to manually hunt for the right RSS link.
100% the current implementation is "RSS as would be desired by newsletter lovers", but there is already the newsletter for that. If I want batching or similar my reader will handle that, I think it would be best just to have items as they happen appear on the feed.
Ah, I didn't know it was a thing! I'll add it to HTML head.
I second this. This would be a great feature
So where is the rss feed of most important news per day?
I know it's not going to be popular, but to cover the cost of running ChatGPT on that many articles, I made it a part of a premium subscription: https://www.newsminimalist.com/premium#rss
Do you need realtime results, or is an ongoing queue of article analysis good enough? Have you considered running your own hardware with a frontier MoE model like deepseek v3? It can be done for relatively low cost on CPU depending on your inference speed needs. Maybe a hybrid approach could at least reduce your API spend?
source: I run inference locally and built the server for around $6k. I get upwards of 10t/s on deepseek v3
PS: thank you for running this service. I've been using it casually since launch and find it much better for my mental health than any other source of news I've tried in the past.
Thank you so much! Always glad to see long-time readers.
There was a period when I considered switching to an open-source model, but every time I was ready for a switch, OpenAI released a smarter and often cheaper model that was just too good to pass up.
Eventually I decided that the potential savings are not worth it in the long term - it looks like LLMs will only get cheaper over time and the cost of inference should become negligible.
Thanks for the reply! This is perhaps not so much a Hacker News type question since this place is very VC focused, but have you considered publishing any papers on your system? I think it would make a fascinating and valuable bit of research.
Or, even farther off the deep-end: have you considered open-sourcing any old versions of your prompts or pipeline? Say one year after they are superseded in your production system?
I don't oppose these ideas, but it's a matter of priorities. There's just many other features and improvements to implement that seem more valuable.
Nice work. Subscribed.
I had a very similar idea a while back. I wanted to rank news by "impact" which might be more concrete than "significance."
For an LLM prompt, it would be something like:
"estimate the number of people who's lives that will be materially changed by this news." and "estimate the average degree of change for those impacted."
Then impact is roughly the product of those two.
Additionally, I want a version that is tailored to me specifically "estimate the degree of change this will have on my life." + context of my life.
Tangentially, I've found that getting ratings out LLMs works better when I can give all options and request relative ratings. If I ask for rankings individually I get different and less good results. Not enough context length to rate all news from all time in one go though. Any thoughts on that? Maybe providing some benchmark ratings with each request could help? Something I'm exploring.
What you're describing is super close to the first version I had!
In the beginning I had 3 parameters: scale (number of people), magnitude (degree of change for those impacted) and additionally potential (how likely is this event to trigger downstream significant events).
The point behind including potential was to separate these two events:
1) A 80 year old dies from cancer 2) An 80 year old dies from a new virus called COVID
This worked roughly well but I kept adding parameters to improve the system: novelty, credibility, etc... The current system works on 7 parameters.
---
I never attempted to give LLM all options and rank them against each other.
1) as you said, for me 20k articles is just too much to fit into context window. Maybe some modern LLMs can handle it, but it wasn't the case for a long time, and I settled on current approach.
2) I don't want the "neighbors" to affect individual article ratings. With the current system I am able to compare news spread over months, because they were all rated using the same prompt.
3) I intentionally avoided giving AI examples, like "evaluate event X given that event Y is 7/10". I want it to give scores with a "clear mind" and not be "primed" to my arbitrary examples.
I like it! How have you thought about news sources getting snippy when you summarise them and don’t send traffic? Not accusatory at all, I’m still unpacking my own opinion on it and wondering how much pushback services will get.
I'd really like to see Top significant news for the past week/month. I'd rather try to read weekly/monthly digest than consume the feed every hour/day. Is that possible?
I think the closest thing to what you want is the newsletter I send 2-3 times a week: https://newsletter.newsminimalist.com/
It's available via RSS too: https://rss.beehiiv.com/feeds/4aF2pGVAEN.xml
I attempted to make a weekly version, but quickly dropped the idea. Over the course of the week articles often became outdated (not just old, but plain wrong).
I found that an optimal newsletter schedule is sending it about every 48-72 hours, depending on how eventful that period was. With this frequency, the articles rarely become outdated, and at the same time it's not too frequent to get tired of.
But that's a thing: if the article becomes outdated during the course of the week - was it even important for me to read? Assuming I'm not a daytrader or anything like that
I generate summaries and custom digests for my feeds— https://taoofmac.com/space/blog/2025/01/12/1730#daily-news-d...
Might take a stab at ranking them as well.
I use this sometimes and it's a nice source of news. But, and this is an honest question, will this get eaten by something like this?:
"ChatGPT, set a daily/weekly task to give me the most significant news. Use this ranking criteria: <input criteria>"
This is so cool. I love this application of LLMs
Thank you!
I think LLMs are really underutilized as a "judgement tool". A couple similar ideas people reached out to me with were: evaluating which pull requests are more significant in a big repo, or which grant applications have more merit.
The LLMs will always make mistakes, but they could work great as the first filter.
you may like @JungleSilicon on twitter.
neato! Curious where you're sourcing the raw feed from. Wire service (AP, Reuters, AFP)? Google news? "the wikipedia current events page"? Something else?
As in every other engineering endeavor, the raw data you start off with has a lot to do with what you end up with, no matter what transforms happen. :)
Thanks! I mainly rely on Paid APIs, but additionally go over some RSS feeds for prominent sources that they don't cover.
Wikipedia current events page was actually one of the reasons for creating this project! I was disagreeing a lot with their selection of news, for example one of the recent entries is:
"Two people are killed and eleven others are injured when a bus flips on its side on a highway near Prenzlau, northeast of Berlin, Germany."
My system gave it a significance score of 1.8: https://www.newsminimalist.com/articles/two-dead-and-four-in...
To your point about raw data - I wasn't expecting there to be so much junk in news. I think I automatically filter out like a third of articles that I get: historic recaps (something cool happened 50 years ago), round-ups (Here's what happened last week), a ton of opinion articles, and AI-slop.
You don't have to use AI for this. Significance has already been figured out by how many news websites a news story has been published on. Which you can infer via RSS feed collection and looking at the title of the news story. The more platforms published the same title, the higher the significance.
These are the most popular news headlines from 136475 news headlines collected from 9503(english only) RSS feeds collected on 2025-01-15.
I'm working on something similar but am thinking of using AI differently. Great job Vadim.
The default sorting is "new first", but perhaps a better first impression would come from selecting "more significant first":
The results seem really bad, multiple entries are for the same thing and others are for very minor events. It looks like the frontpage of any random news site.
Nice. Similar to https://news.ycombinator.com/item?id=35905437 (which summarizes for free)
I think the main differentiator for my project is news selection.
Most other aggregators show news based on 1) relevance, 2) upvotes or 3) coverage.
Relevance-based algorithms tend to put reader into a bubble, where the more they read on a certain topic, the more news they see on that topic.
Upvotes-based algorithms usually bring up a lot of clickbait and drama.
Sorting by coverage doesn't really work either, media often just follows people's interests and churns articles on what is "hot".
For example, last summer, a fight between Zuckerberg and Musk was at the top of most feeds based both on upvotes and coverage. Significance-based algorithm didn't even put it in the top 50.
I understand (also by reading your other responses here). Very nice, I like it. Is the summary feature based on one article (the top one mentioned), or does it combine all articles on that topic? And any roadplan for a native iOS app?
Summaries are done for a single article. I think it's better this way — a summary is a clear reflection of an underlying article, where all statements can be easily tracked to a single original source. That makes LLM hallucinations extremely rare.
No plans for an iOS app, but my site is a PWA so can be installed and look like an app both on Android and iOS.
I kinda like the approach of Boring Report, where it does combine multiple sources into one summary. Still, I like your https://newsletter.newsminimalist.com/archive page. What about using other models instead of GPT-4o mini, like Claude 3 Haiku, Llama 3.1, or Mixtral (which now started a partnership with AFP https://news.ycombinator.com/item?id=42732960).
Most people live outside the US. So maybe change "World" and "Nation". Unless you are focusing on US only.
How can I filter out (block) subjects/words?
Yeah, the category names don't convey the meaning behind them very well. There's no intentional focus on US.
"World" includes everything that talks about two or more countries.
"Nation" is a category with news that touch only a single country, not necessarily US. The current feed is very US-heavy because practically everyone (even non-US sources) is talking about Trump.
Keyword blocking is available on premium: https://www.newsminimalist.com/premium#block-topics
this is really cool. some thoughts - a free trial would make a lot of sense for the subscription. Having the AI break things down into more variables than just significance might be helpful. For example - events vs trends. "Trump shot" vs "Shootings up X% YoY". Forward looking statements vs concrete events "Trump lays out agenda" vs "Trump signs executive order". All of those are significant in different ways. Also newness of the information is important. Top article right now is about Gazan families looking to return home post ceasefire. But that doesn't add much to the genuinely significant news yesterday of the ceasefire itself. As is, I don't get news that seems particularly different from browsing the headlines at 2-3 top newspapers.
Thanks for the deep feedback!
The default feed sorting is done for regular visitors (new first), for evaluating the output you might like the "significant first" more: https://www.newsminimalist.com/?sort=significance
On that list, the ceasefire article is on the second place out of the ~40k articles analyzed.
---
Having more variables is an good idea. I don't have an immediate vision on how to use it in the UI (I want to keep it minimal), but will think more about it.
---
I've been really torn on free trial. I currently offer a refund guarantee, but will add a trial as well soon.
How would this compare "significance" of news about the ceasefire in Israel/Palestine vs, say, a genocide that's large in numbers but smaller in world attention? (edited for clarity, given your reply)
It'd be interesting to see, but I don't fully trust ChatGPT to rate historic events that it already knows are significant. A much more fair comparison would be between events that it doesn't have in it's training data.
But from what I've seen it's pretty indifferent to "sides", it's more focused on raw numbers of people affected and magnitude of the event.
Yeah, this is a great idea. Filtering the ranked output would be even better, especially if you could also exclude content by class (or keyword), as in...
rank by significance:
NOT politics; NOT Trump; NOT republican; NOT democratIt's actually available in paid subscription! Users asked for it a lot, mostly to block the recent political drama in US and Ukraine/Russia, Israel/Palestine conflicts.
Pricing is pretty steep for an aggregator. I know running the LLM is expensive, but maybe there could be a "lite" tier that offers a very basic customization. Cache top scoring results and let people subscribe to one of _n_ pre-computed feeds with more or less news.