Ask HN: Algorithm for searching text with similar meaning
For example, I have a long text that says, "I have to change the screw number 1234," and a short one that says, "screw number 1234 changed." Both were inserted by different people, referring to the same thing. I thought of using an LLM (GPT-4), however, my dataset is too large (millions of entries) and it would be expensive. Is there any other better or good enough way? Thank you.
Read original source
Source record
Recorded metrics
points · 1 votes
2026-11-06 15:30:20.070Z
cumulative since publication
Research analysis
Reported problem
Source account: For example, I have a long text that says, "I have to change the screw number 1234," and a short one that says, "screw number 1234 changed." Both were inserted by different people, referring to the same thing. I thought of using an LLM (GPT-4), however, my dataset is too large (millions of entries) and it would be expensive. Is there any other better or good enough way? Thank you.
Frequency & alternatives
Recorded author: mococa. Source votes: 1 votes (Nov 6, 2026 UTC; cumulative since publication); previous same-scope observation: 1 (Nov 6, 2026 UTC). These observations do not establish demand growth. Recent records to compare: “DeepSeek-V4: beam search leaves HCA/CSA compression caches on the wrong beams” (GitHub Issues, Nov 4, 2026); “[问与答] 纯工具性的软件创业走到头了” (V2EX, Nov 2, 2026). Shared themes are navigation hints; they have not been verified as the same need or independent corroboration.
Demand hypothesis
Treat the reported task or obstacle as an interview lead. A feature request, complaint or search for alternatives does not by itself identify a budget owner or willingness to pay.
Commercial evidence gaps
Missing evidence: distinct affected users, repeated occasions, current workaround, measured time or money cost, purchasing authority and a response to a concrete offer.
Validation decision
Group original reports by the same task and deduplicate authors and reposts. Ask how the task is handled today and what it costs; test a defined offer before claiming paid demand.