ML Koans

Passing through the NAND-gate

June 23, 2021 — June 23, 2021

machine learning
neural nets
Figure 1

1 Case 1

If a neuron drops out in a hidden layer and no-one is there to weigh it, does it influence the prediction?

2 Case 2

What is the softmax of a one-category classifier?

3 Case 3

The SGD optimization was running on the HPC, and two data scientists started an argument.
One said the model moved, the other said the data moved;
they argued back and forth but could not reach a conclusion.
The finance department said, “It is not the data that moves, it is not the model that moves; it is your funding that moves.”
The two data scientists were awestruck.
Figure 2: Listen to the loss function decay asymptotically toward zero.

4 Case 4: AGI

Jõshû asked Nansen, “What is AGI?”
“Ordinary mind is AGI,” Nansen replied.
“Shall I try to seek after it?” Jõshû asked.
“If you try for it, you will become separated from it,” responded Nansen.
“How can I know AGI unless I try for it?” persisted Jõshû.
Nansen said, “AGI is not a matter of knowing or not knowing.
Knowing is delusion; not knowing is confusion.
When you have really reached true AGI beyond doubt, you will find it as vast and boundless as outer space.
How can it be talked about on the level of right and wrong?”
With these words, Jõshû came to a sudden realization.

5 Case 5: Tech support

Yakusan the HPC support officer had not ascended the rostrum for a long time. The institute steward said, “All the staff has been wishing for instruction for a long time. Please, Master, give the noobs a sermon.” Yakusan had the bell rung. The noobs gathered. Yakusan ascended the rostrum and sat there for a while. Then he descended and returned to his room. The institute steward followed him and asked, “You said a while ago that you would give the noobs a sermon. Why didn’t you speak even a word?” Yakusan said, “For neural nets, there are neural net specialists; for physics simulations, there are physics simulation specialists. Why do you have doubts about this old HPC support officer?”

6 Sources