Main Page
<ask default="None yet" format="embedded" limit="2" sort="published" order="desc"> +"+" contains an extrinsic dash or other characters that are invalid for a date interpretation.</ask>
Facebook checking my activity
Facebook locked my account because of unusual behavior. I'm thankful they're checking. I often see obviously spammy behavior on Facebook.
Then they show me my latest posts and comments and ask me which one of these wasn't by me. And they all were by me. There was nothing in the sort of "Oh, I've now seen three of your posts, and you look like a really interesting person. Do you want to be my friend?" or trying to sell NFTs and coins or day trading.
Yeah, no, AI will still take a moment.
Experiment to understand LLMs better
Here’s an experiment I would love to do if I had the resources. Just to start gaining some more understanding of how LLMs work.
- Train an LLM Z on a lot of English text.
- Ensure that the LLM in its response uses correctly the past tense of “go”, “went”, in its responses.
- Ask the LLM directly what the past tense of “to go” is, and expect “went”.
- Remove all sentences / texts from the corpus that contain the word “went”. Add more text to the corpus to make it roughly the same size again.
- Train an LLM A on that corpus.
- Use the same prompts to see what the LLM uses instead of “went”.
- Ask the LLM directly what the past tense of “to go” is. I expect “goed”?
- How many example sentences / texts containing the text “went” does one need to add to the corpus of LLM A and retrain in order for the resulting LLM to get it right. Is one enough? Ten? A thousand?
- Add an explicit sentence ‘The past tense of “to go” is “went”’. to the corpus of LLM A and retrain instead of the implicit training data. Did the trained LLM now get it right? Does it use it right? Does it answer the explicit question correctly?
- Add an explicit sentence to the prompt of LLM A, instead of retraining it. Does it use the word right? Does it answer the explicit question correctly?
If there is some similar work to this out there, or if anyone has some work like this, I’d be very curious for pointers.
P.S.: Also, I would love to see whether people who do research on LLMs could correctly predict the result of this experiment ;)
