Pipeline Long video datasets are challenging to build because of the significant manual effort required to select, watch, understand and annotate long videos with free-form natural language. Answering challenging questions about longer videos is often a multimodal task that may involve listening to the audio track in addition to watching the video. It may also […]
Abstract We propose a novel geometric model to explain the observed redshift of light from distant celestial objects without invoking cosmic expansion or gravitational redshift. By examining the angular geometry between the light source, the observer, and a fixed reference point “above” the observer, we demonstrate how spatial geometry alone can lead to an apparent
Research Published 12 September 2024 Authors Robotics team Two new AI systems, ALOHA Unleashed and DemoStart, help robots learn to perform complex tasks that require dexterous movement People perform many tasks on a daily basis, like tying shoelaces or tightening a screw. But for robots, learning these highly-dexterous tasks is incredibly difficult to get right.
Large Language Models (LLMs) have revolutionized how we interact with information, but grounding their responses in verifiable facts remains a fundamental challenge. This is compounded by the fact that real-world knowledge is often scattered across numerous sources, each with its own data formats, schemas, and APIs, making it difficult to access and integrate. Lack of
LLM-based chatbots’ capabilities have been advancing every month. These improvements are mostly measured by benchmarks like MMLU, HumanEval, and MATH (e.g. sonnet 3.5, gpt-4o). However, as these measures get more and more saturated, is user experience increasing in proportion to these scores? If we envision a future of human-AI collaboration rather than AI replacing humans,
AI-generated images are not just hyper-realistic; they can also be crafted to embody an infinite variety of features, expressions, and aesthetics. A recent example is a post by Moritz Stellmacher on X that showcases a very intriguing human portrait, leaving viewers puzzled about whether it’s a photograph of a real person or an AI creation. This
Science Published 5 September 2024 Authors Protein Design and Wet Lab teams New AI system designs proteins that successfully bind to target molecules, with potential for advancing drug design, disease understanding and more. Every biological process in the body, from cell growth to immune responses, depends on interactions between molecules called proteins. Like a key
When we began studying jailbreak evaluations, we found a fascinating paper claiming that you could jailbreak frontier LLMs simply by translating forbidden prompts into obscure languages. Excited by this result, we attempted to reproduce it and found something unexpected.
Science Published 22 August 2024 Authors David Pfau and James Spencer Note: This blog was first published on 19 October 2020. Following the publication of our breakthrough work on excited states in Science on 22 August 2024, we’ve made minor updates and added a section below about this new phase of work. Using deep learning
Large language models (LLMs) are incredible tools that enable new ways for humans to interact with computers and devices. These models are frequently run on specialized server farms, with requests and responses ferried over an internet connection. Running models fully on-device is an appealing alternative, as this can eliminate server costs, ensure a higher degree
