4 ms·

I can confirm that this (7B) runs nicely on a 24GB MacBook Air M2. The output of my initial test was definitely a bit different than ggreganov's example! The

I can confirm that this (7B) runs nicely on a 24GB MacBook Air M2. The output of my initial test was definitely a bit different than ggreganov's example! The first man on the moon was 39 years old on July 16, 1969. July 16th is the 198th day of the year (199th in leap years) in the Gregorian calendar. There are 168 days remaining until the end of the year. 1561 – France is divided into 2535 circles (French: cercles) for fiscal purposes. 1582 – Pope Gregory XIII, through a papal bull, establishes the Gregorian calendar (Old Style and New Style dates). 1

snickmy 4y ago
The self attention of this model needs some tuning :) It reminds me how sometimes I have stream of consciousness while talking, and I have ADHD.
- gorbypark 4y ago
  Yeah, definitely. The second run was similarly off the rails.. The first man on the moon was 12 years old when the American Revolution began. As a boy, he was horrified by the sight of his mother being whipped. He came to America after surviving the Spanish Inquisition and massacres in Mexico and Brazil. In Texas he married a second-generation German, the daughter of a Confederate soldier who had never heard of the Battle of Bull Run. As a teenager, he fished salmon out of the Columbia River for his Russian Orthodox mother-in-law who said she had seen the sign of the cross appear in the sky before a victory in battle. He followed the death Still impressive for a 7B model that's been slimmed down to 4 bits.
gorbypark 4y ago
As a followup, the 13B model similarly runs great on the 24GB MacBook Air. It seems to perform a lot better (output wise) and about twice as slow per token. Output from 7B "The first man on the moon was ": The first man on the moon was 19-year-old Luna 15-year-old man, so the eyes of the world on him, as he was the first man in the moon after the moon. The second man on the moon was a 45-year-old American man, Neil Armstrong. In 1969, Neil Armstrong, with Aldrin, landed on the moon and set foot on the moon, became a hero to the world. At that time, the United States succeeded in landing on the moon with a manned spacecraft, NASA's Apollo 11 Lunar Module main: mem per token = 14434244 bytes main: load time = 1865.80 ms main: sample time = 259.36 ms main: predict time = 23119.33 ms / 170.00 ms per token main: total time = 25916.14 ms Output from 13B "The first man on the moon was ": The first man on the moon was 38 year old Neil Armstrong (Buzz Aldrin was the second man to walk on the moon, he was the lunar module pilot). What was the first man to land on the moon? Neil Armstrong was the first person to step on the moon on July 20, 1969. What did Neil Armstrong say when he landed on the moon? Neil Armstrong said, "Houston, Tranquillity Base here, the Eagle has landed." What are the facts about Neil Armstrong? Neil Armstrong is an American main: mem per token = 22439492 bytes main: load time = 3692.42 ms main: sample time = 260.61 ms main: predict time = 42439.68 ms / 312.06 ms per token main: total time = 47629.22 ms

snickmy 4y ago

The self attention of this model needs some tuning :) It reminds me how sometimes I have stream of consciousness while talking, and I have ADHD.

gorbypark 4y ago
Yeah, definitely. The second run was similarly off the rails.. The first man on the moon was 12 years old when the American Revolution began. As a boy, he was horrified by the sight of his mother being whipped. He came to America after surviving the Spanish Inquisition and massacres in Mexico and Brazil. In Texas he married a second-generation German, the daughter of a Confederate soldier who had never heard of the Battle of Bull Run. As a teenager, he fished salmon out of the Columbia River for his Russian Orthodox mother-in-law who said she had seen the sign of the cross appear in the sky before a victory in battle. He followed the death Still impressive for a 7B model that's been slimmed down to 4 bits.

gorbypark 4y ago

As a followup, the 13B model similarly runs great on the 24GB MacBook Air. It seems to perform a lot better (output wise) and about twice as slow per token. Output from 7B "The first man on the moon was ": The first man on the moon was 19-year-old Luna 15-year-old man, so the eyes of the world on him, as he was the first man in the moon after the moon. The second man on the moon was a 45-year-old American man, Neil Armstrong. In 1969, Neil Armstrong, with Aldrin, landed on the moon and set foot on the moon, became a hero to the world. At that time, the United States succeeded in landing on the moon with a manned spacecraft, NASA's Apollo 11 Lunar Module main: mem per token = 14434244 bytes main: load time = 1865.80 ms main: sample time = 259.36 ms main: predict time = 23119.33 ms / 170.00 ms per token main: total time = 25916.14 ms Output from 13B "The first man on the moon was ": The first man on the moon was 38 year old Neil Armstrong (Buzz Aldrin was the second man to walk on the moon, he was the lunar module pilot). What was the first man to land on the moon? Neil Armstrong was the first person to step on the moon on July 20, 1969. What did Neil Armstrong say when he landed on the moon? Neil Armstrong said, "Houston, Tranquillity Base here, the Eagle has landed." What are the facts about Neil Armstrong? Neil Armstrong is an American main: mem per token = 22439492 bytes main: load time = 3692.42 ms main: sample time = 260.61 ms main: predict time = 42439.68 ms / 312.06 ms per token main: total time = 47629.22 ms