Thanks for the response ... it might be fine for people with better eyes, or better monitors, but as you can see from the screenshot, the contrast is very low for me.
Love the work, I've saved the link into my system and will be reading it more carefully later. If you'd care to send me an email I'll send you a link to some draft thoughts I've had about knowledge systems. No obligation, obviously.
Hello steve,
I like how you are integrating pytorch and other ML models in your project. Looking forward to it and I hope you the best. Do you have a twitter profile or a mailing list? I found only your email and linked in profile on the website. I want to be updated on this project.
Not OP but main gripe is that it fails to integrate with native environments and for efficiency-orentied note taking app losing that efficiency edge native environments provide is just counter-productive.
Vaporware is better than going nowhere! (Get it...noware...haha).
Congrats on getting started.
I agree with Obsidian - I think that most people forget the maintenance time it takes to build a lifelong Knowledge Management System.
I like your idea - document similarity is a well known area in ML.
Feel free to take my Chrome Extension and use the parts where it tracks key paragraphs in an article (using a user's click/ hover/ attention behaviour) and use that as the corpus for your ML similarity models.
Intuitively it makes more sense to run document similarity on key points/ paragraphs than the whole web page.
If you want the whole web page though, there's code in the Chrome Extension that use's Mozilla's readability lib (https://github.com/mozilla/readability) to purify the web content.
Thanks for the tip on the readability library. I don't have much experience with webdev, so my extension was just saving a local copy of whatever was returned every time the browser made a request, I should be able to cut down on storage space if I can use the readability library to skip saving things like trackers and images.
This blog is a summary of a fun 1 month adventure I had with Knowledge Management Systems and building a POC that I thought had potential. It was inspired by so many of the products I see on Hacker News.
I’ve open sourced all the code + written down some notes on my insights/ architecture. Just a warning, the code leaves much to be desired as this was a mini project over 2-3 weekends.
At the moment, I have 3 key layers of the application.
The first is Chrome Extension which allows data collection. It tracks all sorts of data such as attention, time on page, scroll/ click and hover behaviour. It then sends this data to the Electron app.
The Electron app receives this data through an API and saves it down locally using PouchDB (data structure is compatible with Apache CouchBD and thus allows easy cloud storage).
Within the Electron app, I use an Express/ NodeJS web server to expose endpoints for functionality with the front end/ receiving data from Chrome Extension.
For the front end I use ReactJS.
If you’re interested, please read the full article! There are lots of videos of the application in action.
Built something like that 5 years ago
https://twitter.com/MindDriveCo/status/668518135880916992
Have been working ever since to address many of the issues discussed in this thread and lot more.
On the way to turning Engelbart's Conceptual Framework
https://www.dougengelbart.org/content/view/138
for Augmenting Human Intellect into a Practical (meta) Framework. "Go Meta Young Man' and join the effort to build a Kernel for Open Collective Intellect on the Decentralized Web.
It is not a commercial venture, but wanting to contribute to Web 3 something comparable in impact to Ward Cunningham's Wiki, something that have the potential to trully change the workld as Ward Cunningham idea of the Wiki had.
Primarily
Enduring, Open, Co-evolvable, Bootstrapable, metadesignable, promoting edge-user autonomy, uneclosable collaboration and much more
Forgot to add: It also tracks what you click so it can build relationships between articles/ blogs/ anything else you read on the internet.
For example, if I’m reading an article about big data/ ETL pipelines and then I click on a link from within the article to a resource on machine learning, this relationship should be tracked and digested.
This is neat. I had a similar idea of tracking what we read online in the browser. In addition to just recording time spent and links followed, I would also archive every article read to automatically build up a personal library of the articles themselves. Additionally, light weight note taking on the articles themselves ala hypothe.is
Unfortunately a new baby has drained all my time for such pursuits.
So right now, the Chrome Extension tracks all link clicks as well so the visualisation actually builds relationships between articles/ blogs/ anything else you read on the internet.
I also had another idea that once we build our Knowledge Maps, we should be able to compare and share it will one another.
For instance, I'd love to see what our software engineers around my age/ experience are reading and the insights they are drawing from articles.
Obviously some sort of security/ privacy mechanism will need to be implemented as well.
It is definitely possible to sync with CouchDB. I've just found that the uncertainty around how to do authz/authn (CouchDB recommends, now, that you do this in your own proxy layer) and the difficulties to make sure the entire process works, makes debugging tricky. I've tried with a variety of NodeJS+CouchDB and PouchDB-server and nothing gives me a solution which I don't have to babysit quite a bit.
