Author: Min Wang

General intro https://github.com/alexandrainst/responsible-ai https://ai.google/responsibilities/responsible-ai-practices/ https://www.tensorflow.org/responsible_ai   Open source implementation https://github.com/microsoft/responsible-ai-toolbox https://www.tensorflow.org/responsible_ai/api_docs https://opendatascience.com/15-open-source-responsible-ai-toolkits-and-projects-to-use-today/ Responsible AI Toolkits for AI Ethics & Privacy TensorFlow Privacy TensorFlow Privacy is a Python library that includes implementations…

A poor man C/C++ linter and code coverage (gtests) C/C++ Linter: Cppcheck apt-get install cppcheck cppcheck –enable=all /your_cpp_source_dir   Test Code coverage : gcov/lcov g++ -o main –fprofile-arcs -ftest-coverage main_test.cpp…

Free and open-source software Columnar DB   Database Name Language Implemented in Notes Apache Druid Java started in 2011 for low-latency massive ingestion and queries Apache Kudu C++ released in…

Use cases both provide strong consistance for key/value store. zookeeper use ZAB, etcd use raft, usually one leader. normally use as configure store Zookeeper more like file system https://zookeeper.apache.org/doc/r3.3.6/zookeeperStarted.html bin/zkCli.sh…

The CAP theorem states that a distributed system cannot simultaneously be consistent, available, and partition tolerant No distributed system is safe from network failures, thus network partitioning generally has to be tolerated.[7][8] In…

Raft: paxos hard to understand, new consensus algorithm consensus algorithm:  Leader elections, log replicate https://github.com/etcd-io/etcd/blob/main/raft/README.md   This Raft library is stable and feature complete. As of 2016, it is the most…

Time Series db using cassandra https://docs.datastax.com/en/tutorials/Time_Series.pdf Try it out https://www.datastax.com/try-it-out   cqlsh:demo> create TABLE demo.users3(lastname text, firstname text, time timestamp , primary key(lastname, time)); cqlsh:demo> INSERT INTO users2(lastname, firstname ,…

row-based vs col based db or format row based –> good for OLTP ( transcation),   e.g: cassendra col based –> good for OLAP (? easy to aggreation etc?), druid Parquet…

GRPC   client side msg: header, msg, EOS server side msg:  header, msg, msg,  Trailer over http2 keep live? c++ async or sync? https://grpc.io/docs/languages/cpp/async/ , does it provider more performance than…

Hadoop: Hadoop kubernets MapReduce Spark on K8s Flink stream HDFS S3? any better one Resource manager Yarn/Mesos K8s itself   During its evolution phase, Hadoop provided three main functionalities that…