large language models

Mar 02

Structured Outputs for LLMs

Jeffrey Wang

Back in the early days of LLMs, it was a struggle to get structured output out of models, which made it hard to use the results in a programmatic way -- there was no well-defined interface. That problem was solved a while back, and now most inference systems support JSON schema outputs. As part of...

large language models

Feb 23

Speculative Decoding in LLM Inference

Jeffrey Wang

Running frontier LLMs is slow, but that's the tradeoff we make to get more intelligent output. But if you think about the text (tokens) that an LLM produces (or that a human produces, for that matter), you might have an intuition that a lot of it does not actually require high intelligence....

large language models research paper

Feb 18

Triton Language

Jeffrey Wang

The world of GPU programming for AI has come a long way since I worked on writing a CUDA-based matrix library back in 2009. Both NVIDIA hardware and the CUDA ecosystem have evolved dramatically and are now the basis for the majority of AI compute in the world today (hence the $4T+ market cap)....

large language models research paper gpu performance

Feb 15

Linear Representations and Superposition

Jeffrey Wang

As LLMs become larger, more capable, and more ubiquitous, the field of mechanistic interpretability -- that is, understanding the inner workings of these models -- becomes increasingly interesting and important. Similar to how software engineers benefit from having good mental models of file...

large language models research paper

Feb 08

Quantization-Aware Distillation

Jeffrey Wang

Following up on the last post, I want to write about this new paper from NVIDIA on quantization-aware distillation. We learned about quantization last time, so let's talk about distillation now. Model distillation is the process of transferring knowledge from one model (the teacher) to another,...

machine learning large language models gpu performance

Feb 03

LLM Quantization and NVFP4

Jeffrey Wang

With the rise of large language models and the desire to run them more cheaply and efficiently, the concept of quantization has gained a lot of traction. By representing the weights of the LLM with data types that use fewer bits, you reduce the necessary GPU memory to load the model and the memory...

machine learning large language models gpu performance

Dec 06 2025

Failures of “AI” Promise: Critical Thinking, Misinformation, Prosociality, & Trust

Damien P. Williams

So, new research shows that a) LLM-type “AI” chatbots are extremely persuasive and able to get voters to shift their positions, and that b) the more effective they are at that, the less they hew to factual reality. Which: Yeah. A bunch of us told you this. Again: the Purpose of LLM- type “AI” is...

environment education technology democracy consciousness epistemology phenomenology ai justice ethics teaching facebook fascism data centers uncategorized amazon propaganda pedagogy llm misinformation google data science bullshit audio racism systems openai large language models knowledge my writing fairness disability prejudice values responsibility algorithmic bias gpt authoritarianism misogyny bias equity race neurodiversity algorithmic justice my work social cognition a future worth thinking about feminist ethics invisible architectures of bias technological ethics social construction of science actor-network theory science technology and society biopolitics marginalization environmental ethics implicit bias personhood rights gender social dynamics ableism social shaping of technology sts environmental justice homophobia marginalized lived experiences science and technology studies yingchong wang algorithmic systems harry frankfurt sexism surveillance culture disinformation technoscience compassion generative pre-trained transformer social construction of technology facial recognition neuroethics biomedical ethics jess reia my words damien patrick williams machine ethics disability studies transphobia environmental impacts systemic disparity intersubjectivity invisible architecture of bias philosophy of technology mc forelle biotech ethics will straw embodied cognition

Sep 09 2023

Practical causal ordering: Why weighted-Directed Acyclic Graphs (DAGs) are powerful for causal inference?

msuzen

Preamble Fractal Tree (Wikipedia)A quiet causal inference revolution is underway in industry. We see immense success of transformers deep learning architectures. However their success should also be attributed to causal modelling. Large Language Models (LLMs), specially closed-sourced ones,...

llm nlp causality dag self-attention dags wdags large language models causal inference