2
$\begingroup$

I'm doing a masters in CS that requires me to implement from scratch most of the neural network models and because python libraries aren't applicable to what i want.

The problem is that i don't feel capable of implementing those from scratch. Even the MLP seems very difficult, i always try but get stuck and restart. Meantime, i feel that my supervisor think it's a natural thing and i shouldn't have problems implementing those algorithms.

I know that when we are talking about research, it shouldn't be easy - because anyone else would do it if so - but i'd be glad to receive some advices about this process i am going through: Am I not prepared enough for this or is it an overambitious idea to implement all these networks from scratch? Should i continue this? I've already thought about giving up.

Ps: any good materials (books, videos, etc) focusing in implementation of neural networks are welcome.

Improve this question
$\endgroup$
1
  • $\begingroup$ From someone who did give up, a long time ago, I would recommend sticking with it. Accept feeling in over your head as the first phase; the answers here are giving good advice - keep digging! $\endgroup$
    – user46107
    Feb 17, 2020 at 17:24

2 Answers 2

Reset to default
5
$\begingroup$

I have been implementing a branch and bound solver with heuristics for an NP-hard problem. It got complicated at some points and had to reimplement parts a couple of times. The problem was (I think), that I started implementing with only an intuition about the design and how it looks like. That is bad software engineering and is catastrophic in big project.

So forget your computer for now. Bring a bunch of white papers and a pen. Describe the steps of the algorithm precisely. Design the program, decide the data structures and how they will communicate. Decide where data should be kept and how it should be commuted. When you have the details drawn on papers, coding is just a matter of writing it down and doing some optimization.

Another good advise is to start by writing the simplest version you can imagine. Implement only what you need to have, in the simplest way you can think of. No optimization, no copy/reference calls thinking just get it to run correctly at first. Once you are at that point you have something to deliver, start making it better. Check where you are doing stuff naively where you can do it better. Find all copies where you could have just done it on the original structure. Because if you kept optimizing and adding features while implementing you will never finish. There will always be something to add.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ These are valuable advices. Thank you. I've been trying the same way you did. Sometimes my algorithm threw erros and i was fixing it in a 'trial-and-error' manner. That's not good because the more complex the project became, the harder it was to understand what to do. $\endgroup$
    – heresthebuzz
    Feb 14, 2020 at 15:18
2
$\begingroup$

(Not enough reputation to comment, so writing here.)

  1. Unless your algorithms are secret, please post your algorithms here. Maybe someone (not me though) can find a library for you. Maybe someone can tell how long does it take to implement it.

  2. Use Git and GitHub. You can rollback bad code with this.

  3. Always write tests. This helps against regressions as you update/optimize your algorithm. Moreover, code without testing are almost certainly broken.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thanks for the kind advices. I am dealing with Ensemble Learning (Machine Learning) but none of available libraries apply to my project. I have no experience coding AI algorithms from scratch, and they are very frustrating if you don't know 100% what you are doing. $\endgroup$
    – heresthebuzz
    Feb 17, 2020 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.