Experimenting with ReLU
ReLU -> Leaky ReLU
Leaky ReLU -> Weird Asymptotic ReLU
Escaping local minima with exploration
Learning rate could be reduced for the very early turns. Proportional to how late the turn is?
alpha * (turn_number/total_number_of_turns)
in that way, the closer you are to the end, the closer you are to the max learning rate and the closer to the start, the closer to the minimum learning rate