The Appliance


Our next version of analytic software for computer performance data will have a totally new architecture and design, to support large or small customer accounts, using same stock base software. From Perl to Lua, from a time series data store to a very powerful in cache data structure server, all these changes were made to support more devices, efficiently use the computing power and deliver value. So what we did:
  • we dropped Perl and FastCGI
  • we dropped RRDtool
  • we started to use Lua programming language
  • we moved to OpenResty for a fast and robust HTTP processing
  • we switched to Redis for in cache memory statistics
  • and we are still designing a new exploratory data module, for direct interaction with raw data
And the results are amazing. We have a more powerful architecture which lets us build nice ready data appliances for large data-center customers and small and medium sized businesses.

We are using the Data-Driven Document JS library, the D3, a fresh and powerful data visualization library, based on standard HTML, SVG and CSS. So there are lots of new things versus our old analytic release 0.70. And we expect to have the first demo somewhere in late October for certain key accounts.

During all our development process we been heavily reading and researching in cache memory data structure servers, web analytic, real-time data processing. Two reference books we are using heavily for our software development and data streaming:



"Redis In Action", by Josiah L. Carlson
a must have about Redis and data analytic software implemented in Python or Lua. The author is a master in Redis and he is frequent contributor to Redis community. Many thanks Josiah for your support and advices for our analytic software product.








Another good book about real-time web analytic software is "Real-Time Analytics", by Byron Ellis a nice description of data analytic software technologies, methods and techniques to develop and support such applications. It does include Redis database but the others too: MongoDB, Cassandra.






Comments

Post a Comment

Popular posts from this blog

Asus Zenbook and FreeBSD 11

Image
This is a short description how I got running FreeBSD 11-current on my Asus Zenbook UX32VD laptop. Im very happy with the current setup but of course it is room for improvements in many areas. Having DTrace, ZFS and the other goodies makes FreeBSD a real good candidate for a mobile environment. Last Updated: 16 March 2015 Zenbook UX32VD Configuration CPU: Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz Memory: 6 GB RAM Storage: SanDisk SSD i100 24GB, Samsung SSD 840 PRO 500GB Video: Intel HD Graphics 4000, Nvidia GT620M Display: 13" IPS Full HD anti-glare Wifi: Intel Centrino Advanced-N 6235 3 x USB 3.0 port(s)  1 x HDMI  1 x Mini VGA Port  1 x SD card reader Note: The laptop's internal hard drive, has been replaced by a Samsung SSD.
Read more

Raspberry Pi and Redis

Image
This summer we did something amazing: we took our enterprise appliance, a very powerful server and we tried to run same software, same analytic kernel and all the other things in something smaller and lot simpler, like this: Light, mobile appliance And we did it ! Being powered by OpenResty and Lua we were able to size our appliance easily and be up and running quickly. Low factor, low power consumption, very cheap, these ARM based devices are getting more and more attention and become more popular nowadays: Banana Pi - A Highend Single-Board Computer 4 x Raspberry Pi + Arduino = UDOO What ? Analytics on a Raspberry PI ... !? Yep. First of all, we wanted to experiment with ARM based boards using few number of hosts and few data messages per host as input. The immediate trouble was that we should fit in 512MB RAM with all our software and customer data. Then these boards don't run on latest Intel specs but something like 700 Mhz CPU as a system on chip wi
Read more

Web Performance, tell me the story

Image
Suppose you would like to know how your web site, behaves to clients from two different geo locations: for example, Finland and Ireland regarding response time. How quickly can you see and understand visually what's going on and how things are. Do you need to improve something ? Change your network provider ? Tune your application(s) ? Spend more money.  Start from a top level view, understanding the big picture and having the possibility to dive and analyze at the detail level.  Finally, can you visually tell the story of your web application as in a short chart ? Enter Kronometrix Web Performance ... Finland Something happened between 1:00 - 1:15 AM We can see more than one request affected by this event Overall we see all requests are executing fast, below 0.5 seconds And we have some exceptions, some are taking more than 0.5 seconds Ireland Same things from Ireland It is different, requests usually take longer to execute
Read more