5 ms·

Reverse engineering the Ravensburger TipToi pen

100 points by cl3misch 2y ago

croisillon 2y ago
it's funny this toy seems to exist only since 2010, i used to have something very similar around 1991, 1992
- croisillon 2y ago
  ah, questron it is https://news.ycombinator.com/item?id=42338088 https://news.ycombinator.com/item?id=42338088
reid 2y ago
Looks very similar to Korea’s Saypen! My daughter loves it.
bhouston 2y ago
Does this work for the V-Tech Bugsby? or LeapFrog LeapReader? I was wondering if they are licensing the same technology base.
- Suppafly 2y ago
  The leapfrog stuff is what came to mind for me, I remember we had similar things when my kids were little. Honestly, they never seemed to work very well, but I assume the newer versions of such things are better.
- Karliss 2y ago
  Based on this https://www.youtube.com/watch?v=O4FUZcF_IC4 https://www.youtube.com/watch?v=O4FUZcF_IC4 video LeapRader seems to use similar technology in the sense that they both are based on grid of points with offsets. Although at least by eye the timing/anchor dots don't seem to match the exact scheme described in TipToi reverse engineering repo. Also didn't see any obvious Sonix chips in LeapReader, at least in the teardown pictures I saw. Looking at the Sonix website (the company providing camera+decoder chip solution used by TipToi) https://www.sonix.com.tw/category-en-956 https://www.sonix.com.tw/category-en-956 they seem to have at least 6 generations of the code with varying amount of encoded bits. Some of the related patents https://patents.google.com/patent/US6548768B1/en https://patents.google.com/patent/US6548768B1/en (Anoto ~1999/2003) https://patents.google.com/patent/US8061611B2/en https://patents.google.com/patent/US8061611B2/en (Canon ~2006/2011) https://patents.google.com/patent/US20070246547A1/en https://patents.google.com/patent/US20070246547A1/en (Sonix ~2006/2009) https://patents.google.com/patent/US10614333B2/en https://patents.google.com/patent/US10614333B2/en (Sonix ~2015/2016) they seem to describe slightly different variations of data encoding. Searching for Anoto (company that made one of the first patents) leads to a bunch more products using this technology. List of 400+ patents https://patents.google.com/patent/US6548768B1/en#citedBy https://patents.google.com/patent/US6548768B1/en#citedBy citing the Anoto 1999/2003 patent also gives a list of companies working on related products or technology. There are even some from LeapFrog but those seem to mostly cover practical application and UI aspects of a product using the optical position/code detection system. Overall feels like Anoto is focusing more on a system which decodes the points into X/Y coordinates and covers whole page with contiguous pattern, but Sonix more on having codes which give specific ID although they also one version which gives XY position similar to Anoto. That might be partially a workaround to make the encoding schemes different enough not to infringe each others patents. That doesn't really give a good answer to your question, but If I had to guess Leapfrog is probably buying their technology somewhere else possibly from Anoto. There is also one similar product using Grolier name, which seems to use tech from Sonix just like TipToi so potentially compatible, at least the codes not necessarily the book format.
  - bhouston 2y ago
    Thank you for the detailed reply based on your research!
xxmarkuski 2y ago
Entropia is the local chapter from Karlsruhe of the Chaos Computer Club. The Gulaschprogrammiernacht (GPN) is their local version of Congress located in HfG and ZKM, a college and a great museum.
Suppafly 2y ago
The puzzle company?
- scary-size 2y ago
  Ravensburger makes all kinds of things, books, toys, and puzzles.
amaurose 2y ago
I saw Joachim talk about Tiptoi years ago here: https://www.youtube.com/watch?v=GzXtgR73icg https://www.youtube.com/watch?v=GzXtgR73icg IIRC, thats the video with the talking calculator demo...
unwind 2y ago
I had never heard of this, and the repo doesn't provide a lot of background ... it seems to be a handheld (!) pen with interactive point-for-audio features, for 2+ year-olds. Here [1] is the Ravensburger product page in German and here [2] is some toy seller's page. [1]: https://www.ravensburger.de/de-DE/produkte/tiptoi https://www.ravensburger.de/de-DE/produkte/tiptoi [2]: https://www.playpolis.com/ravensburger/tiptoi https://www.playpolis.com/ravensburger/tiptoi
- ce4 2y ago
  The pen's main function is to point at things in the TipToi books and its integrated speaker explains the thing being pointed at, or plays a song or whatever the book is about. There is a huge catalogue of books for e.g. learning names of things in the kitchen, at a farm, fairy tales and even languages etc. It's great stuff and works well There are different models and the newest one has a record feature as well as wifi (for asset downloads, versions before need a download companion app on the PC/mac). Edit: fix typos
- netsharc 2y ago
  The git repo feels like German bureaucracy, one needs to dig around to find the info.... https://github.com/entropia/tip-toi-reveng/wiki/PEN-Optical-ID-and-codes https://github.com/entropia/tip-toi-reveng/wiki/PEN-Optical-... TL; DR: the tip of a pen has a sensor, the books have dot patterns overlaid on top of image areas, the patterns are decipherable into (hexa-)decimal code, and with a script file the codes do different actions (set a variable, play an audio file).
rob74 2y ago
I had the TipToi's precursor as a kid: https://www.erinnerstdudich.de/70er/elexikon-von-ravensburger/ https://www.erinnerstdudich.de/70er/elexikon-von-ravensburge... (scroll down to the comments for more pictures). This used a much simpler device - basically it consisted of two electrical probes connected with a wire, one of them containing batteries and a light bulb, and it worked together with a board with contact pads with hidden interconnections. Then there were several sheets where you had to find the numbers in the picture corresponding to the labels printed along the right side of the sheet. I also like the honesty of the text on the box saying that it's "electrical", when the temptation to (incorrectly) use "electronical" instead must have been great...
- peepee1982 2y ago
  Ah! I salvaged one of those from the local waste collection center and it was an easy fix to make it work again! I had a Questron pen as a child and loved it, also made by Ravensburger.
  - croisillon 2y ago
    i had a questron too! much closer to the tiptoi than to elexikon https://news.ycombinator.com/item?id=42338848 https://news.ycombinator.com/item?id=42338848
    - peepee1982 2y ago
      Yes, wireless even. Pretty nifty.
globalise83 2y ago
As an immigrant father of 2 small children, living in Germany for the past few years, I have found that Germany has some real "hidden champions" amongst the tech-enabled learning products space. This Tip-toi product and associated ecosystem is one of them, and another is Tonies, which is a kind of audio player in a nice cushioned box that uses cartoon-like figures to choose which audio books to play.
- joezydeco 2y ago
  I just saw these Tonies at Target (USA) last night, it looks like that store got the exclusive distribution rights for the US. $99 for the starter box? Wow.
- peepee1982 2y ago
  I am a father of three small ones in Austria. The Tonies are cute, but I find them just so frivolously expensive! My children loved to listen on audio books on CDs (can be rented from libraries or bought for cheap) or on an old 1st gen iPod.
  - ce4 2y ago
    I got ours all used in bundles. The box itself is rather repairable (broken silicone ears, dead sd card, dead battery) and there are sometimes very good deals on Kleinanzeigen classified ads. Also what helps: resale value is great, so after the Tonies have had their time and your kids become older... just sell them off.
  - bonzog 2y ago
    The thing I dislike most about Tonies is that they don't licence the original recordings for lots of the movie/TV themed characters. They licence the songs and then record them in-house and goodness me some of them are disappointing if you are familiar with the real soundtracks. The Moana one has the same two singers doing impressions of every character.
  - tjungblut 2y ago
    Get a flipper, you can simply download and emulate all of them
niemandhier 2y ago
The biggest selling point of the pen for me is, that while it works with special kinds of books, the books are not tied to any account. You can get a book from the library, buy a used one or borrow one from a friend, it will work with your pen. This is so refreshingly different from all those subscription models, that I keep buying the books and once my kids grew out of them I know I can gift them away.
- moring 2y ago
  Note that while it seems that you need an account to download the audio files, this isn't actually the case. It's just the stupid UI on their web site that makes it look like that. If you find your way through that UI, you can get the audio files without logging in, and the pen announces itself to the host computer as a thumb drive -- just copy the files into the root directory. At least that was the case 2-3 years ago.
  - heeen2 2y ago
    it is still the case and easy to find if you google for the title of the book plus the file extension (gme). They are also all listed here https://service.ravensburger.de/tiptoi%C2%AE/tiptoi%C2%AE_Audiodateien https://service.ravensburger.de/tiptoi%C2%AE/tiptoi%C2%AE_Au...

100 points by cl3misch 2y ago

croisillon 2y ago

it's funny this toy seems to exist only since 2010, i used to have something very similar around 1991, 1992

croisillon 2y ago
ah, questron it is https://news.ycombinator.com/item?id=42338088 https://news.ycombinator.com/item?id=42338088

reid 2y ago

Looks very similar to Korea’s Saypen! My daughter loves it.

bhouston 2y ago

Does this work for the V-Tech Bugsby? or LeapFrog LeapReader? I was wondering if they are licensing the same technology base.

Suppafly 2y ago
The leapfrog stuff is what came to mind for me, I remember we had similar things when my kids were little. Honestly, they never seemed to work very well, but I assume the newer versions of such things are better.
Karliss 2y ago
Based on this https://www.youtube.com/watch?v=O4FUZcF_IC4 https://www.youtube.com/watch?v=O4FUZcF_IC4 video LeapRader seems to use similar technology in the sense that they both are based on grid of points with offsets. Although at least by eye the timing/anchor dots don't seem to match the exact scheme described in TipToi reverse engineering repo. Also didn't see any obvious Sonix chips in LeapReader, at least in the teardown pictures I saw. Looking at the Sonix website (the company providing camera+decoder chip solution used by TipToi) https://www.sonix.com.tw/category-en-956 https://www.sonix.com.tw/category-en-956 they seem to have at least 6 generations of the code with varying amount of encoded bits. Some of the related patents https://patents.google.com/patent/US6548768B1/en https://patents.google.com/patent/US6548768B1/en (Anoto ~1999/2003) https://patents.google.com/patent/US8061611B2/en https://patents.google.com/patent/US8061611B2/en (Canon ~2006/2011) https://patents.google.com/patent/US20070246547A1/en https://patents.google.com/patent/US20070246547A1/en (Sonix ~2006/2009) https://patents.google.com/patent/US10614333B2/en https://patents.google.com/patent/US10614333B2/en (Sonix ~2015/2016) they seem to describe slightly different variations of data encoding. Searching for Anoto (company that made one of the first patents) leads to a bunch more products using this technology. List of 400+ patents https://patents.google.com/patent/US6548768B1/en#citedBy https://patents.google.com/patent/US6548768B1/en#citedBy citing the Anoto 1999/2003 patent also gives a list of companies working on related products or technology. There are even some from LeapFrog but those seem to mostly cover practical application and UI aspects of a product using the optical position/code detection system. Overall feels like Anoto is focusing more on a system which decodes the points into X/Y coordinates and covers whole page with contiguous pattern, but Sonix more on having codes which give specific ID although they also one version which gives XY position similar to Anoto. That might be partially a workaround to make the encoding schemes different enough not to infringe each others patents. That doesn't really give a good answer to your question, but If I had to guess Leapfrog is probably buying their technology somewhere else possibly from Anoto. There is also one similar product using Grolier name, which seems to use tech from Sonix just like TipToi so potentially compatible, at least the codes not necessarily the book format.
- bhouston 2y ago
  Thank you for the detailed reply based on your research!

xxmarkuski 2y ago

Entropia is the local chapter from Karlsruhe of the Chaos Computer Club. The Gulaschprogrammiernacht (GPN) is their local version of Congress located in HfG and ZKM, a college and a great museum.

Suppafly 2y ago

The puzzle company?

scary-size 2y ago
Ravensburger makes all kinds of things, books, toys, and puzzles.

amaurose 2y ago

I saw Joachim talk about Tiptoi years ago here: https://www.youtube.com/watch?v=GzXtgR73icg https://www.youtube.com/watch?v=GzXtgR73icg IIRC, thats the video with the talking calculator demo...

unwind 2y ago

I had never heard of this, and the repo doesn't provide a lot of background ... it seems to be a handheld (!) pen with interactive point-for-audio features, for 2+ year-olds. Here [1] is the Ravensburger product page in German and here [2] is some toy seller's page. [1]: https://www.ravensburger.de/de-DE/produkte/tiptoi https://www.ravensburger.de/de-DE/produkte/tiptoi [2]: https://www.playpolis.com/ravensburger/tiptoi https://www.playpolis.com/ravensburger/tiptoi

ce4 2y ago
The pen's main function is to point at things in the TipToi books and its integrated speaker explains the thing being pointed at, or plays a song or whatever the book is about. There is a huge catalogue of books for e.g. learning names of things in the kitchen, at a farm, fairy tales and even languages etc. It's great stuff and works well There are different models and the newest one has a record feature as well as wifi (for asset downloads, versions before need a download companion app on the PC/mac). Edit: fix typos
netsharc 2y ago
The git repo feels like German bureaucracy, one needs to dig around to find the info.... https://github.com/entropia/tip-toi-reveng/wiki/PEN-Optical-ID-and-codes https://github.com/entropia/tip-toi-reveng/wiki/PEN-Optical-... TL; DR: the tip of a pen has a sensor, the books have dot patterns overlaid on top of image areas, the patterns are decipherable into (hexa-)decimal code, and with a script file the codes do different actions (set a variable, play an audio file).

rob74 2y ago

I had the TipToi's precursor as a kid: https://www.erinnerstdudich.de/70er/elexikon-von-ravensburger/ https://www.erinnerstdudich.de/70er/elexikon-von-ravensburge... (scroll down to the comments for more pictures). This used a much simpler device - basically it consisted of two electrical probes connected with a wire, one of them containing batteries and a light bulb, and it worked together with a board with contact pads with hidden interconnections. Then there were several sheets where you had to find the numbers in the picture corresponding to the labels printed along the right side of the sheet. I also like the honesty of the text on the box saying that it's "electrical", when the temptation to (incorrectly) use "electronical" instead must have been great...

peepee1982 2y ago
Ah! I salvaged one of those from the local waste collection center and it was an easy fix to make it work again! I had a Questron pen as a child and loved it, also made by Ravensburger.
- croisillon 2y ago
  i had a questron too! much closer to the tiptoi than to elexikon https://news.ycombinator.com/item?id=42338848 https://news.ycombinator.com/item?id=42338848
  - peepee1982 2y ago
    Yes, wireless even. Pretty nifty.

globalise83 2y ago

As an immigrant father of 2 small children, living in Germany for the past few years, I have found that Germany has some real "hidden champions" amongst the tech-enabled learning products space. This Tip-toi product and associated ecosystem is one of them, and another is Tonies, which is a kind of audio player in a nice cushioned box that uses cartoon-like figures to choose which audio books to play.

joezydeco 2y ago
I just saw these Tonies at Target (USA) last night, it looks like that store got the exclusive distribution rights for the US. $99 for the starter box? Wow.
peepee1982 2y ago
I am a father of three small ones in Austria. The Tonies are cute, but I find them just so frivolously expensive! My children loved to listen on audio books on CDs (can be rented from libraries or bought for cheap) or on an old 1st gen iPod.
- ce4 2y ago
  I got ours all used in bundles. The box itself is rather repairable (broken silicone ears, dead sd card, dead battery) and there are sometimes very good deals on Kleinanzeigen classified ads. Also what helps: resale value is great, so after the Tonies have had their time and your kids become older... just sell them off.
- bonzog 2y ago
  The thing I dislike most about Tonies is that they don't licence the original recordings for lots of the movie/TV themed characters. They licence the songs and then record them in-house and goodness me some of them are disappointing if you are familiar with the real soundtracks. The Moana one has the same two singers doing impressions of every character.
- tjungblut 2y ago
  Get a flipper, you can simply download and emulate all of them

niemandhier 2y ago

The biggest selling point of the pen for me is, that while it works with special kinds of books, the books are not tied to any account. You can get a book from the library, buy a used one or borrow one from a friend, it will work with your pen. This is so refreshingly different from all those subscription models, that I keep buying the books and once my kids grew out of them I know I can gift them away.

moring 2y ago
Note that while it seems that you need an account to download the audio files, this isn't actually the case. It's just the stupid UI on their web site that makes it look like that. If you find your way through that UI, you can get the audio files without logging in, and the pen announces itself to the host computer as a thumb drive -- just copy the files into the root directory. At least that was the case 2-3 years ago.
- heeen2 2y ago
  it is still the case and easy to find if you google for the title of the book plus the file extension (gme). They are also all listed here https://service.ravensburger.de/tiptoi%C2%AE/tiptoi%C2%AE_Audiodateien https://service.ravensburger.de/tiptoi%C2%AE/tiptoi%C2%AE_Au...