New Horizons

This is a blog about the evolution of my thought process about building websites to maintain and disseminate information on inventory over the past three years. It has been a long journey from tinkering with databases and javascript and finally discovering the zen of csv files and python.

How does one manage inventory? There is no denying that everyone has this problem here. Even after commissioning special storage cabinets and labelled shelves , boxes, pouches and keeping spreadsheets on them, we were at a loss.

In spite of all that work of organising, inventory still has been a game of darts. We still had issues like sending the wrong parts to field. Field personnel and us fumbling on the part names, matcodes, manufacturing part numbers, purchase orders. Then there were other problems - like differences in packaging, similar names with different uses and the list goes on.

And we were back at how does one manage inventory. If you have noticed, questions like these have a way of starting slowly, seeming like a wrestle, followed by periods of being swept under the rug and one day, ending in the monster suddenly re-surfacing in intimidating forms.

Hyperbole aside , we noticed that there were several facets to the same problem. Let me see, if I could list all of them.

1. Not everyone uses a certain spare and hence not every one knows every thing. This is a Scope Problem.

2. Not everyone uses the same terminology. This is a Language Problem.

3. A person who knows and uses a spare part, might not necessarily be the best at describing it . This is a Language Problem too.

So what do you do when you decide to deal with the problem ? Like any good accountant - you look for the ground truth and build from there. It started with a simple and unoriginal idea. Descriptions are causing all the confusion, what if we could see a picture. May be, magically , this inventory problem would be solved.

So, for the next few weeks, our technicians religiously catalogued all our inventory all the while taking pictures of everything. For a while, the store looked like a forensics lab.

Initially, we used a laptop with a spreadsheet for data collection and a mobile camera for pictures.

Later, we graduated to using a simple google form. [ I assure you we gave due regard to the confidentiality & Integrity of the information ]

Now that we had all the information, our next step was to look for ways to make it available to everyone in the department. Our requirements were simple

1. Everyone in the department can search for a spare part and find a picture, current inventory , location and any other meta data.
2. The store keeper can add new spares and update inventory.
3. If you are not a store keeper, you cannot make any changes to the inventory.

Right around this time, I had finished reading 'Eloquent Javascript' and a few books and tutorials on the MEAN stack including 'Web Application Development using MEAN stack'.

So charged with the problem and a means to solve it, I did the following which I present to you as Paradigm #1.

Paradigm #1 :

Build a full featured web site for tracking inventory. One where you could add, remove, update inventory. One with a login window, running on our local network.

So that's what we built.

1. A website built using HTML, CSS, Javascript using the Angular 2 framework. I relied heavily on Bootstrap for styling. The website had three pages. Add, Update, Search. The search page would let you type in a keyword and it returned matches for matcode, part number, description, meta tags etc.
2. All the data was stored in a No SQL database called MongoDB. Schema was really simple, one to two nests.
3. Express.js Server. All the javascript code was heavily borrowed. We had created simple REST APIs and middleware for the Create-Read-Update-Delete loop. We tried integrating passport.js for authentication but eventually gave up. It had some 'session' issue that we could not solve.
4. Node.js — Well although I mentioned this in the end, this was the basis for everything. My familiarity of working with Node.js in one of my previous projects really helped. [More on that in a different post.]

It look us about 3 week - one to two hours on each day , of tinkering got it running. Mostly everyone was happy. Except the guy who staked his life on the efficacy of excel sheets. This was uncharted territory for him and mostly he was worried that he would get a few weird columns in the excel sheet. We used mongoexport to export the database into excel sheet.

Well, looking back, the 'photo' idea seemed useful. As always, the person who appreciated it the most was the one who faced the most problems on a minute by minute basis, THE STOREKEEPER.

In its entirety, these are problems paradigm 1 solved , aka PROs :

• Availability
• Easy Inventory management
• Visibility

CONs of Paradigm 1 :

• Javascript Web App Development, especially Angular 2 is overkill for this application for a small user base.
• Each feature like search, pagination, mongodb integration took us time.
• Huge codebase. We literally spent many afternoons trying to solve a tiny bug.
• You cannot make batch updates to inventory. I mean you could if we wrote another page for that. Think of a basic Javascript TODO with a submit button.
• You need two different applications — a client app and a server app.

After this adventure, I also built a single terminal biometric attendance system using javascript.

Fast forward to March 2020, I began to take obsessive interest in data science. I entered the world of rectangular data aka dataframes. I was rapidly learning data manipulations and visualisations using Python packages.

Let me take a moment and acknowledge the delight of python.

For people coming from C, Java, Javascript, they are taken aback by the simplicity of syntax. Python one-liner codes are so popular, and there's so many of them, people have written books on them. Here's one. I can't resist but show you the python code for swapping two variables. ⇒ a,b = b,a. I could tell about list and dictionary comprehensions too , but you'll figure it. They are simple and elegant. If there's one thing I'd want you to take away from this, it would be : LEARN PYTHON.

In fact, let me introduce you to the guiding principles of Python language.

• Beautiful is better than ugly.
• Explicit is better than implicit.
• Simple is better than complex.
• Complex is better than complicated.
• Flat is better than nested.
• Sparse is better than dense.
• Readability counts.
• Special cases aren't special enough to break the rules.
• Although practicality beats purity.
• Errors should never pass silently.
• Unless explicitly silenced.
• In the face of ambiguity, refuse the temptation to guess.
• There should be one—and preferably only one—obvious way to do it.
• Although that way may not be obvious at first unless you're Dutch.
• Now is better than never.
• Although never is often better than right now.
• If the implementation is hard to explain, it's a bad idea.
• If the implementation is easy to explain, it may be a good idea.
• Namespaces are one honking great idea—let's do more of those!

A popular library for data manipulation in python is Pandas. You can import data from csv, xlsx, images, sql db, api, pdf into pandas and then filter, search and visualise. For example, if you have list of your inventory, it is really easy to do statistical analyses. If you want to know what you can do with pandas ? Look here

Getting back to our inventory problem, our application started misbehaving. Unresponsive database requests. Turns out a lot of packages we used to build our inventory app have been deprecated. We were living under a rock.

Now with these new found super powers, I started to rethink the inventory app problem. We already had the data. Our problem was to somehow use python and pandas and built a new website so that everyone got access.

Enter 'streamlit'. I think it was truly serendipity that brought us together. One lazy Sunday, I read a tweet about how they needed an equivalent of streamlit for Julia. Julia a high performance and scalable data analysis language. It claims to combine the speed and scalability of C with 'Python'y syntax. This tweet stoked my interest in streamlit and I spent the next few hours being blown away by the simplicity of it. With streamlit, you could build 'data' apps i.e websites backed by any of your favourite data containers like csv, excel, sql etc. One of its many great features is , it runs on python. Data scientists have been using streamlit to build and show case their data experiments. If you want to see what is possible with streamlit, go here. [have patience, it takes a while to load]

This is already too long for a single read, let me cover my streamlit experience in the next post. Stay tuned. 

Happy New Year to everyone.

I'm Jayanth and I work with Logging Services, Mumbai. Welcome to the first post of my new blog 'New Horizons'.

I decided a while back that I'm gonna write more in 2021. Partly because I keep having all of these ideas I want to share and they have a way of slipping away . And mostly because, I like the feeling of being routinely surprised by something I've written a while ago and the fresh perspective it brings. Besides, I've been mulling over the reciprocity of the triad of writing, reading , thinking and this blog seems like both an exercise in the triad and a creative outlet.

I hope to write about my current interests and obsessions - Note taking , Data Science, Web Technologies , Privacy & Logistics - not necessarily in that order. As my first post, I wanted to share the principles of building a life changing productivity system I learned from Ali Abdaal.

So here goes.

All of us absorb a lot of interesting information from emails, blogs, videos, books, podcasts etc. I've always felt that most of this information does not reach its usefulness stage. And an extreme case in this is the proverbial out of sight, out of mind. You probably know people who have a million browser tabs open all the time, not closing them for the fear of forgetting what they just consumed. Our brains have a way of holding on to information that is most important to the current predicament. And there is no mistaking that the predicaments are plenty. If you're like me, who handles many disparate tasks to the point of turning into a context switching junkie, you must be thinking about a better way to organise everything you learn and catalogue it in a way that is meaningful / useful in the long term. All of this while reserving our brain's capacity to concentrate on the most important tasks at hand. Think of a personal wikipedia of information & insights and more.

Any efforts in this direction could be dubbed as 'building a second brain'.

This is repository of information and insights that you could fall back on - to make decisions, to draw inspiration and to get stuff done. In the long term, this becomes a source of abundance that you could start your projects with, instead of starting with a blank canvas. The last part I think is immensely useful, given there only so much CHRONOS that goes around. 'Second Brain' is a super charged productivity / knowledge management system. Its about creating a repository of information that you could use to complement your brain. This is productivity system that Ali Abdaal claims to have changed his life. The overarching principle behind building a second brain and gaining the most out of it, is the acronym - CODE : Capture - Organise - Distill - Express. Capture all the information you finding interesting. This could be your own ideas or the ideas that you learn from content you consume. Capturing could be in various ways - voice memos, excel sheets, note taking apps etc. Catalogue all the ideas and give them context. Understand the essence of the captured information. Summarising the content , writing about it is a great way to churn out the essence. And finally the only way you own the information is by making something of it. This could be a blog, a personal project or any creative endeavour.

With due regard to the above overarching principle, there are 10 fundamental principles of building a second brain :

1. Borrowed Creativity

   • Creativity is all about remixing stuff. It is about taking stuff that's already been made and adding your own new perspective to them.
   • Its like what Pablo Picasso said , 'Good artists copy, great artists steal'
   • It's one of the key ideas of second brain system since organising your knowledge helps in finding connections that may have not existed / discovered earlier.
   • Besides, its easier and quicker to make connections using a repository of information than trying to visualize in your head.

2. The capture habit

   • Your brain is to have ideas, not store them. Capture all the ideas you have, coz your brain might not hold them on for long. These could be ideas that you get when you were jogging, taking a shower. Note these ideas down
   • Create mechanisms to capture ideas. For example, use a digital assistant on your phone while driving , voice memo apps.
   • I get most of my creative ideas during or right after some physical exercise, so I intend to use Siri to capture them.

3. Idea recycling

   • Ideas are re-usable.
   • Keep track all of the content you are creating and all the ideas that are being used in them.All the blogs, websites, apps , emails and any creative endeavours are to be documented so that they could be reused somewhere down the line.These ideas then become building blocks that you could probably re-use for newer stuff.
   • For example, the proposal you recently made to your boss could be taught as a course. Or a code block from a web app for inventory could be used for a new website you might build.

4. Projects over categories

   • Associate all the information and learnings to a particular project instead of categories. For example, it is more beneficial to associate a new blog you read on logistics to your shipments project rather than adding it to a folder called Logistics.
   • Such categorization doesnot help because it is not directedly useful and also it might lead to loss of context over time.

5. Slow burns instead of heavy lifts

   • A heavy lift is like picking up that one task and pausing everything else until you finish it. It could be thought of , as a series way of handling tasks. For ex : Going to a cabin the woods for three weeks to write a book. Or dedicating an entire week for just building a new feature into a website.
   • Slow burns are like having multiple projects at the same time , but being handled slowly in parallel. For example, doing a couple of courses and building a product specification, writing a project proposal, all in parallel and each task in small chunks and progression. So overtime, Instead of doing 1 big project one at a time, you could do 10 different projects all at the same time, slowly until they complete.
   • Slow burns are beneficial in the sense that they would let you capture all the things that you learn along the way and use them to complete the tasks. Overtime, you could read, listen and watch ideas and come up with ideas of your own and use these to further the slow burn projects.Instead of a heavy lift, through a slow burn , you could slowly capture ideas from the internet and catalogue them in your second brain
   • Slow burns are beneficial when you have an effective way of capturing all the ideas, notes and material in one central location and somehow linking this repository to the something that tracks the progress of each burn. Also, if the information is organized in one place, it takes less time to consolidate them and apply them to the project.

6. Start with abundance

   • Never start a task like writing an proposal / email with a blank page.
   • If you had a second brain, you could start with abundance of articles, notes, inspirations that you have annotated and collected over time. You could use these bits to conjure interesting arguments / content . And ultimately, this could make a really good article, blog etc.
   • This is exactly like sifting through your brain for ideas / past experiences and you are basically starting any task with the wealth of information that you have decided that its worth saving in the past.

7. Intermeddiate Packets

   • All the tasks that you do are made up of intermeddiate packets. For example, any procurement proposal / webpage specification has certain blocks that are interchangeable and useful for several others.
   • Organise and create such re-usable intermeddiate packets.
   • This would not only make the task easier , it would aslo help starting with a task.
   • Ensure that your second brain is built in a way that makes it easy to think in terms of tasks as intermeddiate packets

8. You only know what you make

   • You only internalize the stuff that you create. That's what affects your lives and is probably useful in the long term. It doesnot matter the number of podcast that you listen to , the movies you watch , the books you read unless at the very least you write summaries / reviews, you use them to build things, write intermeddiate packets, catalogue them into your second brain
   • Always make notes of all the content that you consume.

9. Don't assume intelligence on the part of your future self

   • The way to think of your second brain is that the effort you put in now would help your future self.
   • Dont assume too much intelligence in your future self, create context for everything that you record.
   • For example, if you're writing a daily log about the activites of a particular day with the objective of using them write your self appraisal comments at the end of the year, record in a manner that explains the content and context well, assuming no knowledge.

10. Keep your ideas moving

    • This is the anti perfectionist take on the entire building a second brain.
    • Do not fall into the trap of building the perfect note taking, habit tracking, project structure.
    • The most important thing is the output. It's not about the perfect note taking system, its about improving the system overtime and focusing on the output.

Note that these ideas are not mine at all. This is an endeavour in 'you only know what you make'. If you prefer the video format, check out Ali Abdaal's youtube channel.

My personal choice of second brain was Microsoft Onenote in 2019 and 2020 where I extensively catalogued all my work and personal knowlege. This helped me craft some interesting emails and wrap my head around my routing tasks and workflows. Recently, I spent a few weekends tinkering with Notion because of some glitches with OneNote and I'm convinced that it is the best note taking / knowledge management solution out there for the flexibility and organisation capabilities. Feel free to contact me if you want to discuss more about the second brain idea or just setting up your Notion workspace.

Wish you happy days ahead.