COS711 Assignment 1
Due date: four September 2022, at 23h30
1 Common directions
This task is theoretical, and can take a look at your understanding of the backpropagation algorithm. You need to submit a single pdf doc containing your solutions to the questions offered. Notice: the task is designed to check your potential to derive the load replace equations for arbitrary loss and activation capabilities. Thus, you’ll free marks by skipping over steps. Be sure your derivations are readable, notation is right, and the steps (together with simplifications) are clear.
The report can be checked for plagiarism utilizing Turnitin, and ought to be submitted via the ClickUp system. You’re suggested however not required to typeset your report in LaTeX.
2 Deriving backpropagation (25 marks)
A feed-forward neural community is ready as much as have an enter layer of measurement I, a hidden layer of measurement H, and an output layer of measurement O. The next activation capabilities are employed within the neurons on every layer:
• Enter layer: identification, f(x) = x
• Hidden layer: Softplus, f(x) = ln(1 + ex)
• Output layer: Modified Elliott,
Hidden and output models are summation models, i.e. x within the activation capabilities above refers back to the web enter sign, thus the output sign of a neuron j is yj = f(netj) = f(Pwkjyk). Bias sign is shipped to all hidden, in addition to all output models. Assume the target operate E is used, outlined for a single enter information sample as:
the place yi is the output of the i-th output unit, ti is the goal output of the i-th output unit, ln refers to pure logarithm, s1 and s2 are scalar values such that s1 + s2 = 1, maxE1 and maxE2 are the utmost values produced by E1 and E2 over the info set earlier than coaching, respectively.
Reply the questions beneath:
1. Derive the replace rule for the non-bias weights between the hidden and the output layer. Showall steps, together with simplifications. (10 marks)
1
2. Derive the replace rule for the non-bias weights between the enter and the hidden layer. Showall steps, together with simplifications. (10 marks)
three. Will the bias weight replace guidelines differ from the non-bias weight replace guidelines? Derive theupdate rule for the bias weights related to the hidden layer. Present your working. (5 marks)
2
