Our Research
Current Research Avenues
1. Predictive Vision-Language Models for Efficient Open-World Generalist Agents
We are developing advanced vision-language models that can efficiently navigate and understand open-world environments. Our approach focuses on predictive capabilities, allowing these models to anticipate and adapt to new scenarios without extensive retraining.
2. Data Free LLM Distillation (10-100x Size Reduction) via Internal Activity Prediction
We implement a novel approach to knowledge distillation for LLMs, where a smaller student LLM learns to activity patterns within a larger, pre-trained teacher model.Instead of using the original training data, the method feeds random noise as input to both models and trains the student to match the teacher's internal activation patterns across corresponding layers. This data-free distillation technique allows for potentially more generalized knowledge transfer and doesn't require access to the original dataset, though it may sacrifice some task-specific performance in exchange for broader behavioral mimicry.
3. Neural Activity Prediction for Dynamic Model Adjustment Post-Training
We're exploring methods to dynamically adjust trained models based on future neural activity patterns, expanding on our prior plasticity research found below. This research aims to create more flexible and adaptive AI systems that can evolve their behavior in response to new information or changing environments without the need for complete retraining.
Published Research
Neuro-Inspired Plasticity for Biologically Realistic Self-Adaptation of Neural Network Weights
Authors: R. Kalahasty
Published in: 2023 IEEE International Conference on Development and Learning (ICDL)
Here, we look to biological studies to find the governing rules of plasticity in the PFC - competitiveness, memory, and correlation - to create a biologically plausible implementation of plasticity called Hybrid Plasticity. We implement it in continuous time recurrent neural networks (CTRNNS) completing simple working memory tasks. We show that the implementation of plasticity increases the adaptability of the working memory process within networks, while also resulting in a significant decrease in active neurons within the network indicating higher efficiency.
Research Philosophy
At o37, our research is guided by the principles of the Prefrontal Cortex's predictive capabilities. We believe that by mimicking these natural cognitive processes, we can create more intuitive, adaptable, and powerful AI systems that can truly understand and interact with the world in ways similar to human cognition.