Review of 2018

Review of 2018.

Year 2019 comes soon, now it’s time to review the whole year 2018. This year I learn and focus on a programming language other than R, Python. Thus, projects that mentioned in this blog are almost realised by Python. In the following, I’ll talk about:

  • Working in retailing
  • Self-learning
  • Blog
  • Personal project

Working in retailing

(All sensitive data in this part is either simulated or hidden by mosaic.)

Reporting industrialization

Last year we used a data integration software and accomplished reporting automatization with interface, but this kind of manipulation cannot satisfy our needs. So this year we have an important revolution on reporting industrialization: realising ETL by python and industrializing by gitlab runner. And of course, we converted all lagacy job-processes to this new approach.

Thanks to this revolution, I got how to connect to teradata database and SQL Server by python, had a better and clearer understanding of numpy and pandas’s manipulation, generated and formatted excel worksheets by openpyxl, visualized data with suitable graphs, sent reports/alerts by e-mail, and the last one, which is also the most valuable step, autorise all steps mentioned before by gitlab runner.

Self-learning

Like I mentionned at the begining of this blog, this year I focus on Python programming, for both professional projects and self-learning. In this way, I can consolidate self-learning knowledge during working, and can also advance my professional projects with the knowledge that I learnt after working. This year I learnt with a new approach instead of online courses, I learnt by reading book and writing unit test in python for ensuring my understanding of numpy, pandas and data manipulation like groupby, aggregation, data wrangling and time series.

Moreover, mastering data manipulation methods is not enough, visualization can alive the data. So I learnt how to make graphs with matplotlib. Considering different plots are suitable for differents situations, I explored their use cases and wrote my first blog series, Matplotlib series.

Plot type Use cases
Bar chart Compare categorical data. Comparisons among discrete categories.
Line chart Track changes over time.
X-axis displays continuous variables. Y-axis displays measurement
Pie chart Show percentage or proportional data.
Display data that is classified into nominal or ordinal categories.
Scatter plot Show the relationship between two variables
Treemap Less than 10 branches. Positive values. Space for visualisation is limited.
Venn diagram Show logical relations between different groups (intersection, difference, union).
Area chart Show or compare a quantitative progression over time.
Radar chart Comparing two or more items or groups on various features or characteristics.
Examining the relative values for a single data point.
Word cloud Depicting keyword metadata (tags) on websites.
Delighting and provide emotional connection.
Lollipop plot Showing the relationship between a numerical variable and another numerical or categorical variable.
Histogram Estimating the probability distribution of a continuous variable (quantitative variable).
Organizing large amounts of data, and producing a visualization quickly, using a single dimension.

Blog

Thanks to you, my blog’s traffic increases 102% with respect to 2017. 20 new blogs are written, for R, python, matplotlib, openpyxl, gitlab runner, regular expression and PEP 328. I’m pretty glad if my blogs can help you and solve the problem for you. Some of you wrote me a feedback that I really appreciate.

Google analytics 1

Google analytics 2

Blog feedback

In the next year, I plan to write some blogs about machine learning algorithm and try to improve UI of my blog, since the color combination is not that pretty ;p

Personal project

My husband and I create a small application which can help us to understand and to use reasonably personal finance. For this moment, it’s realised on python and google sheet. If you interest in detail, you can find it here.

Don’t hesitate if you want to ask a queston or write a comment, it’s welcome!!

Happy New Year! Hope to see you in 2019!

Reference

Image “reviewing”