Say you want to read handwritten Devanagari on a phone, offline, in a village with no signal. You can't run a 17-million-parameter network on that phone. You need something small enough to fit, and you need to know it still works. That's the question Barnamala tests: not whether a bigger model can be built, but how small one can go before the numbers stop holding up.
The benchmark is DHCD — 46 handwritten Devanagari characters, a test set of 13,800 images. The model to beat is MallaNet, a published 17.32-million-parameter network. Barnamala's answer is a 1.11-million-parameter student, trained by having a committee of bigger "teacher" models vote on the training examples, then compressing what they agreed on into one small network. On the test set: MallaNet gets 40 of 13,800 wrong. The small model gets 36.6 wrong on average, across five separate training runs. Fewer errors, 15.6 times fewer parameters.
Fewer errors on one test set doesn't prove the small model is actually better — both numbers sit close to a floor every model in the study hits, and the gap could just be noise. So the team ran the test built for exactly that question: McNemar's test, which pairs the specific images each model got right or wrong against each other. The result: p = 0.345. In plain terms, the test cannot say the small model beats the big one — and it cannot say it's worse, either. No significant difference was detected. That's a real result, and a narrower claim than it sounds like: an absence of evidence against equality is not proof of equality.
What's underneath that number is more interesting than the headline. Eleven of the 13,800 test images are missed by every student run the project trained — all fifteen of them, across three different setups — and by MallaNet too. That's a shared wall. Whatever is wrong with those eleven images — bad handwriting, a mislabel, genuine ambiguity — no amount of extra training fixes it on this benchmark. The team calls it a label-noise floor. Nobody has sat down and adjudicated each of those eleven cases by hand, so it's a working theory, not a settled fact.
So what actually happened: a model 15.6 times smaller than MallaNet is not detectably different from it, on a saturated benchmark, using a training recipe that turns out not to depend much on the distillation step that inspired it. The supervised student — no teachers, no distillation, just ordinary training — is also not detectably different from MallaNet on its own, p = 0.701. That's a smaller, stranger result than "distillation works," and it's the one the data actually supports.