Category Archives: Uncategorized

Wittgentsein famously stated that “the meaning is the use”. Unfortunately no analytical explanation of this statement is provided in the posthumously published Philosophical Investigations. Wittgenstein’s point seems to be that language is not appropriately approached through logical or analytical means, … Continue reading →

Behaviorist psychologists refused to talk about mental states or internal computations in human brains on the grounds that everything should be understood in terms of stimulus-response relationships. On the other hand linguists and cognitive psychologists assume the existence of internal … Continue reading →

For a couple years now Michael Douglas and I have been contemplating a meaning representation — simple sentence graphs — based on breaking sentences down into simple sentence components. Over the last week or so we have been playing with … Continue reading →

A recent NOEMA essay by Jacob Browning and Yann LeCun put forward the proposition that “an artificial intelligence system trained on words and sentences alone will never approximate human understanding”.  I will refer to this claim as the grounding hypothesis … Continue reading →

This spring I had a disagreement with an old friend about AGI. They claimed there was essentially no chance of AGI arriving in the next, say, fifty years. I have always said we just don’t know. They also wanted a … Continue reading →

As in previous years, teaching my course on the fundamentals of deep learning has inspired some blog posts. This year I realized that VAEs are non-parametrically consistent as models of the observed data even when the encoder is held fixed … Continue reading →

While teaching reinforcement learning I kept asking myself what AlphaZero teaches us about RL. That question has lead to this post.  This post generalizes AlphaZero to a larger class of RL algorithms by reinterpreting AlphaZero’s policy network as a belief … Continue reading →

Modeling human commonsense reasoning has long been a goal of AI.  From a 1998 interview with Marvin Minsky we have Marvin Minsky: We need common-sense knowledge – and programs that can use it. Common sense computing needs several ways of … Continue reading →

Last week I saw Noah Smith at a meeting in Japan where he mentioned that BERT seemed related to pseudo-likelihood.  After some thought I convinced myself that this observation should lead to a consistency theorem for BERT.  With an appropriate … Continue reading →

I just read  VQ-VAE-2  (Vector-Quantized – Variational AutoEncoders – 2) by Razavi et al. This paper gives very nice results on modeling image distributions with vector quantization. It dominates BigGANs under classification accuracy score (CAS)  for class-conditional ImgeNet image generation.   For reasons listed below … Continue reading →