Machine learning & systems

Manos Theodosis

I work on large language models. From architecture to distributed training and performance engineering.

Currently researcher at Zyphra· US

LLM architecture

Frontier-level architectures: optimal MoEs, expert balancing, quantized training.

Distributed training

Scaling pretrain to 1500+ GPUs: experience in both AMD and NVIDIA.

Performance engineering

Hardware-forward AI design: respect compute and memory constraints.

In the past

  1. Amazon intern·US 2021 audio·dsp·neural nets
  2. National Technical University of Athens researcher·GR 2018–2019 math·optimization·neural nets

Selected research

01 Equivariance

Symmetry in neural networks

Symmetries satisfy constraints and help optimization.

A wireframe sphere and torus A sphere and a torus drawn as simple wireframe surfaces, inspired by the curved domains in the paper. The geometry is schematic; no vector fields or energy measurements are shown. SPHERETORUS
Coordinate-free machine learning.

02 Structured representations

Sparse representations

Representations are controlled by inductive biases.

Grouped atoms and sparse atoms A simplified illustration of the learned-representation comparison. Three outlined rows group variations of zero, four, and eight with distinct proportions and stroke shapes on the left, with a few subtle tilts. Nine stroke-like shapes each occupy their own box on the right. These are symbolic redrawings, not actual learned weights or a new experiment. GROUPED ATOMSSPARSE ATOMS
Biases control representation learning.

03 Tropical geometry

Piecewise-linear geometry

Non-conventional algebras increase expressivity.

Two intersecting tropical surfaces An oblique drawing of the max-plus surface f(x,y) = max(x, y + 2, 7) and the min-plus surface g(x,y) = min(x + 5, y + 7, 9). The ivory and teal surfaces pass through each other along the light six-sided outline. Solid shaded facets hide the parts behind them; the teal center emerges through the ivory surface. Simplified from the intersecting tropical halfspaces in Figure 6. MAX-PLUS SURFACEMIN-PLUS SURFACE
Shapes are efficiently represented.

Publications & talks

Scholar
Publications8
  1. Theodosis·Ba·DehmamyConstructing gauge-invariant neural networks for scientific applications ICML workshops·2024·Link
  2. Tasissa·Theodosis·Tolooshams·BaDiscriminative reconstruction via simultaneous dense and sparse coding TMLR·2024·Link
  3. Theodosis·Helwani·BaLearning group representations in neural networks arXiv·2024·PDF
  4. Theodosis·BaLearning silhouettes with group sparse autoencoders ICASSP·2023·PDF
  5. Maragos·Charisopoulos·TheodosisTropical geometry and machine learning Proceedings of the IEEE·2021·PDF
  6. Maragos·TheodosisMultivariate tropical regression and piecewise-linear surface fitting ICASSP·2020·PDF
  7. Theodosis·Tolooshams·Tankala·Tasissa·BaOn the convergence of group-sparse autoencoders arXiv·2021·PDF
  8. Theodosis·MaragosTropical modeling of weighted transducers algorithms on graphs ICASSP·2019·PDF
Talks & presentations4

A little background

I got my PhD in Computer Science from Harvard, working with Demba Ba at CRISP. I studied ECE at the National Technical University of Athens under Petros Maragos.

I was born in Crete and grew up in Athens. Outside of work I enjoy board games, video games, horror films, and nature. I dabble in landscape photography and I collect quotes.

Alongside research

Mentoring

MentoRes

Konstantinos Kallas and I mentored underrepresented students during their grad school applications, pro bono, through MentoRes.

Life caught up so MentoRes is mostly obsolete, but I still am happy to help; dms are open.