FeedCollection

http://www.hack.lu/news.rdf returned no data, or LWP::UserAgent is not available.

http://a.6f2.net/svnweb/index.cgi/adulau/rss/ returned no data, or LWP::UserAgent is not available.

http://www.michael-noll.com/feed/ returned no data, or LWP::UserAgent is not available.

2025-02-13

• 15:50 UTC Language Models Do Addition Using Helices – Language Models Do Addition Using Helices wtf: Mathematical reasoning is an increasingly important indicator of large language model (LLM) capabilities, yet we lack understanding of how LLMs process even simple mathematical tasks. To address this, we reverse engineer how three mid-sized LLMs compute addition. We first discover that numbers are represented in these LLMs as a generalized helix, which is strongly causally implicated for the tasks of addition and subtraction, and is also causally relevant for integer division, multiplication, and modular arithmetic. We then propose that LLMs compute addition by manipulating this generalized helix using the “Clock” algorithm: to solve a+b, the helices for a and b are manipulated to produce the a+b answer helix which is then read out to model logits. We model influential MLP outputs, attention head outputs, and even individual neuron preactivations with these helices and verify our understanding with causal interventions. By demonstrating that LLMs represent numbers on a helix and manipulate this helix to perform addition, we present the first representation-level explanation of an LLM’s mathematical capability. Tags: llms helices trigonometry magic weird ai papers arithmetic addition subtraction
• 15:50 UTC Random Numbers at 200 Gbit/s – Random Numbers at 200 Gbit/s Very cool trick from Tony Finch; using the PCG random number generator, AVX or NEON vector instructions on modern CPUs allow generation of multiple RNG states at once, in parallel Tags: rngs avx neon vector-instructions cpu parallelism pcg random randomness hacks
• 12:40 UTC CarbonRunner – CarbonRunner “Carbon-aware infrastructure to optimize your CI/CD workflows” — “A multi-cloud CI/CD Github Actions Runner that shifts your workflows to the lowest CO2 regions. 90% Greener. 25% Cheaper. 1 line of code. Zero Effort. ?” (Via Dryden Williams) Tags: green sustainability carbon github ci cd workflows development via:climateactiontech
• 12:40 UTC Critical Ignoring as a Core Competence for Digital Citizens – Critical Ignoring as a Core Competence for Digital Citizens “Critical ignoring” as a strategy to control and immunize one’s information environment (Kozyreva et al., 2023): Low-quality and misleading information online can hijack people’s attention, often by evoking curiosity, outrage, or anger. Resisting certain types of information and actors online requires people to adopt new mental habits that help them avoid being tempted by attention-grabbing and potentially harmful content. We argue that digital information literacy must include the competence of critical ignoring—choosing what to ignore and where to invest one’s limited attentional capacities. We review three types of cognitive strategies for implementing critical ignoring: self-nudging, in which one ignores temptations by removing them from one’s digital environments; lateral reading, in which one vets information by leaving the source and verifying its credibility elsewhere online; and the do-not-feed-the-trolls heuristic, which advises one to not reward malicious actors with attention. We argue that these strategies implementing critical ignoring should be part of school curricula on digital information literacy. Good to give names to these practices, since we’re all having to do them nowadays anyway… (Via Stan Carey) Tags: psychology trolls media kids internet literacy attention critical-ignoring ignoring papers via:stancarey
• 12:10 UTC LinuxPDF – LinuxPDF It’s Linux, running inside a PDF file. “The humble PDF file format supports JavaScript – with a limited standard library, mind you. By leveraging this, [vk6] managed to compile a RISC-V emulator (TinyEMU) into JavaScript using an old version of Emscripten targeting asm.js instead of WebAssembly. The emulator, embedded within the PDF, interfaces with virtual input through a keyboard and text box.” (via Fuzzix) Tags: via:fuzzix linux pdf hacks emulation javascript emscripten tinyemu

2025-02-11

• 14:30 UTC Undergraduate Upends a 40-Year-Old Data Science Conjecture – Undergraduate Upends a 40-Year-Old Data Science Conjecture This is a great story; bonus that it’s a notable improvement for the humble hash-table data structure: Krapivin was not held back by the conventional wisdom for the simple reason that he was unaware of it. “I did this without knowing about Yao’s conjecture,” he said. His explorations with tiny pointers led to a new kind of hash table — one that did not rely on uniform probing. And for this new hash table, the time required for worst-case queries and insertions is proportional to (log x)^2 — far faster than x. This result directly contradicted Yao’s conjecture. Farach-Colton and Kuszmaul helped Krapivin show that (log x)^2 is the optimal, unbeatable bound for the popular class of hash tables Yao had written about. Paper here — https://arxiv.org/abs/2501.02305 . Tags: data-structures hash-tables cs programming coding papers optimization open-addressing
• 14:10 UTC PleIAs/common_corpus – PleIAs/common_corpus This is great to see: Common Corpus is the largest open and permissible licensed text dataset, comprising 2 trillion tokens (1,998,647,168,282 tokens). It is a diverse dataset, consisting of books, newspapers, scientific articles, government and legal documents, code, and more. Common Corpus has been created by Pleias in association with several partners and contributed in-kind to Current AI initiative. The dataset in its entirety meets the requirements of the Code of Conduct of the AI Act and goes further than the current requirements for data transparency. It aims to set a new standard of openness in AI, showing that detailed provenance at a granular document level is a realistic objective, even at the scale of 2 trillion tokens. Tags: ai llms open-data open-source pleias common-corpus corpora training ai-act

2025-02-10

• 16:30 UTC Government agency removes spoon emoji from work platform amid protests – Government agency removes spoon emoji from work platform amid protests lol. “On Wednesday, employees at the Technology Transformation Services division of the [U.S. government’s General Services Administration] reportedly unleashed a torrent of spoon emojis in the chat that accompanied an organization-wide, 600-person video conference with new leader Thomas Shedd, a former Tesla engineer.” … Workers embraced the digital cutlery to protest the Trump administration’s “Fork in the Road” resignation offer.” Tags: forks spoons funny protest us-politics emojis
• 10:50 UTC Are better models better? – Are better models better? This is very interesting, on the applicability and usefulness of generative AI, given their inherent error rate and probabilistic operation: Asking if an LLM can do very specific and precise information retrieval might be like asking if an Apple II can match the uptime of a mainframe, or asking if you can build Photoshop inside Netscape. No, they can’t really do that, but that’s not the point and doesn’t mean they’re useless. They do something else, and that ‘something else’ matters more and pulls in all of the investment, innovation and company creation. Maybe, 20 years later, they can do the old thing too – maybe you can run a bank on PCs and build graphics software in a browser, eventually – but that’s not what matters at the beginning. They unlock something else. What is that ‘something else’ for generative AI, though? How do you think conceptually about places where that error rate is a feature, not a bug? (Via James Tindall) Tags: errors probabilistic computing ai genai llms via:james-tindall
• 10:30 UTC Woof.group vs the OSA – Woof.group vs the OSA The UK’s new Online Safety Act law is extremely vague, extremely punitive, and has Fediverse operators Woof.group very worried — Ofcom carefully avoided answering almost all of our questions. They declined to say whether ~185 users was a “significant number”. Several other participants in Ofcom’s livestreams also asked what a significant number meant. Every time, Ofcom responded obliquely: there are no numeric thresholds, a significant number could be “small”, Ofcom could target “a one-man band”, and providers are expected to have a robust justification for deciding they do not have a significant number of UK users. It is unclear how anyone could make a robust justification given this nebulous guidance. In their letter, Ofcom also declined to say whether non-commercial services have target markets, or whether pornography poses a “material risk of significant harm”. In short, we have no answer as to whether Woof.group or other Fediverse instances are likely to fall in scope of the OSA. Do we block pre-emptively, or if and when Ofcom asks? This is the ethical question Woof.group’s team, like other community forums, have been wrestling with. Ofcom would certainly like sites to take action immediately. As Hoskings warned: “Don’t wait until it’s too late. That’s the message. Once you do get the breach letter, that is when it is too late. The time doesn’t start ticking from then. The time is ticking from—for part five services, from January, part three from July.” Tags: woof.group fediverse mastodon social-media uk osa laws ofcom porn blocking

• Superlinear Returns
• How to Do Great Work
• How to Get New Ideas
• The Need to Read
• What You (Want to)* Want
• Alien Truth
• What I've Learned from Users
• Heresy
• Putting Ideas into Words
• Is There Such a Thing as Good Taste?
• Beyond Smart
• Weird Languages
• How to Work Hard
• A Project of One's Own
• Fierce Nerds