kozikow

40 ms·

1. 391.▲
   by kozikow 10y ago

   Imagine there would be "docker engine for running in production" outside of control of docker inc, with backing from the rest of orchestration industry. It may be considered as a tool for compelling docker inc. to play more nicely
2. 392.▲
   by kozikow 10y ago

   I have a processing pipeline that does crawling, indexing and some analytics on the crawled data - I am building a vertical-focused search engine focused on catering for some specific kinds of companies. Usually my cluster is very small, un
3. 393.▲
   by kozikow 10y ago

   I see two reasons for this: - If you don't use google container engine (hosted kubernetes, also known as GKE), kubernetes have a reputation of being hard to set up. I am running on GKE, so I can't comment too much. - It's har
4. 394.▲
   by kozikow 10y ago

   I went through a few iterations for logging, but now I settled on using built-in GKE logging. Stdout logs from my containers are picked up by kubernetes and forwarded to stackdriver. Since it's just stdout I do not create too much lock
5. 395.▲
   by kozikow 10y ago

   I am using kubernetes instead of docker swarm to orchestrate docker images and all points mentioned in this article do not apply. My cluster is small - I have <100 machines at peak, but so far it feels ready for prime time. There are par
6. 396.▲
   by kozikow 10y ago

   I recently moved my django running on docker to google container engine. Basically by following this tutorial you can be up in 15 minutes: https://cloud.google.com/python/django/container-engine . k8s picks up you
7. 397.▲
   by kozikow 10y ago

   Everyone tries to get the 100% of the orchestration pie. Pie gets smaller as barrier of entry is higher due to the fragmentation. If those companies would focus on securing position in the pie, rather than owning the whole pie, the pie woul
8. 398.▲
   by kozikow 10y ago

   Recommendation engine based on (occurs in same base post) or (suggested by the same HN user) could be interesting as well.
9. 399.▲
   by kozikow 10y ago

   I wish that docker would adapt more of a Unix philosophy and focus on doing one thing well. Why does everyone have to compete with everyone rather than create set of tools that work well together? I see docker-machine and docker-swarm as di
10. 400.▲
    by kozikow 10y ago

    http://kubernetes.io/docs/user-guide/secrets/ works well for me. Not every problem have to be solved by Docker, and in a way that's good and more aligned with Unix philosophy.
11. 401.▲
    by kozikow 10y ago

    I version my docker images. Inside the Dockerfile I usually install the latest version (i.e. pip install without specifying version). By single version bump of my image version I can trigger update of everything. If I check out an old commi
12. 402.▲
    by kozikow 10y ago

    Addressing some criticism from various comments: > Python 2 vs 3 It is mostly over. All important libraries are accessible on 3. It is clear that if you are starting from scratch you should go with 3. There are good tools to help with th
13. 403.▲
    by kozikow 10y ago

    > Having used Rubygems and bundler pip feels like taking an enormous step back. Personally I moved away from relying on language specific tools to do more than just "install library X" and manage my deployment and dependencies
14. 404.▲
    by kozikow 10y ago

    IMO except a few small sub-domains like neural networks, we have much more computing power than ideas for things we can do with it or people that can implement those ideas. Supporting people side of the business - salaries, benefits or offi
15. 405.▲
    by kozikow 10y ago

    I wish that stackdriver documentation would be more clear that you should go for sending logs through fluentd, or you are going to have bad time. I have been using python wrapper over the stackdriver http api. To fix a few frustrations I ha
16. 406.▲
    by kozikow 10y ago

    In my current project, I have my own mini-infrastructure, where each machine have a "watchdog" process that picks up items from global queue, and calls "docker run" with the item it picked up from the queue. Docker knows
17. 407.▲
    by kozikow 10y ago

    I recently just plugged python logger from https://github.com/GoogleCloudPlatform/gcloud-python#google-... into my python codebase. Configuration took 15 minutes, I have nice dashboard showing recent logs, I can log ei
18. 408.▲
    by kozikow 10y ago

    Yeah, it feels too much like an advertisement of their product. All points mentioned are related to their product and I can think of some important aspects that were not mentioned, but are not addressed by their product.
19. 409.▲
    by kozikow 10y ago

    I am curious why you used the user-closeness model. Matrix factorisation won the Netflix prize: http://dl.acm.org/citation.cfm?id=1608614 . What made you go for user closeness?
20. 410.▲
    by kozikow 10y ago

    I added some graphistry and clustering analysis to the link.
21. 411.▲
    by kozikow 10y ago

    99% chance that it's not possible and it won't be possible soon. You can take a look at the table at https://bigquery.cloud.google.com/table/the-psf:pypi.downloa... and check out tabs "Schema" and &
22. 412.▲
    by kozikow 10y ago

    d3 tends to be very low level and there are a few successful libraries built on top of it. Co-occurrence graph library certainly would be popular. In my visualisation I've spent the most time fiddling with JavaScript and currently it&
23. 413.▲
    by kozikow 10y ago

    Does any of those libraries would support: - Mouse over to highlight neighbours - Zoom in and out - Browser based - Ideally "hide this cluster" functionality I wanted to add some of those features using d3, but the weekend ran out
24. 414.▲
    by kozikow 10y ago

    I agree. I was planning to add more things, but the weekend ran out. By using graphistry.com I will add: - Zooming in and out. - Mouse over the package to highlight nodes with an edge. - Cluster the packages and let users remove clusters th
25. 415.▲
    by kozikow 10y ago

    Do you have any idea for algorithm that would find "alternative packages to X", e.g. matplotlib would return seaborn/bokeh/ggplot/etc. ? Some rough idea would be high correlation of their neighbour weights, but low
26. 416.▲
    by kozikow 10y ago

    Thanks! I will join this data with pypi stats.
27. 417.▲
    by kozikow 10y ago

    I will try out the http://graphistry.com/ today after work. Zooming and mouse over will help extract information. In the second section of the post I posted some information I extracted. It was hard to balance number of nod
28. 418.▲
    by kozikow 10y ago

    Looks great! Today after work I plan to try out version with PMI and https://github.com/graphistry/pygraphistry .
29. 419.▲
    by kozikow 10y ago

    I plan to do the analysis for other languages soon.
30. 420.▲
    Visualizing relationships between Python packages(kozikow.com)

    63 points by kozikow 10y ago|24 comments