Major or Minor Design Flaw

Recently, I am experiencing that my laptop’s performance has reduced and it need restart. To sort the problem, I uninstalled all unnecessary softwares, cleaned the temp folders, cleaned the registry, defragment the memory, etc. But still no visible effect in performance. Further investigation lead to me that as laptop start heating up its performance degrades. So I, checked its fan, it was working fine. Then why laptop is getting heated. Any hardware failure. NO.

Suddenly I realized that rubber pads in the base of laptops are not there so air is not flowing. So where are the rubber pads? The rubber pads were glued to slots. It is strange that designers of laptops (mine was $1200) have not provided sufficient quality of rubber pads or something which can ensure the air flow? When I examined laptops from various vendors, all have same design flaw, except Apple which has provided elevated notch in base, no need of pad.

Are laptop designers not taking lessons from successful or some patent issue!!!

Some interesting concepts for Software Architect and Designer

1. Convention over Configuration (aka Coding by convention)
2. Don't Repeat Yourself (DRY, aka Single Point of Truth)
3. Code Reuse
4. The KISS principle (Keep it Simple, Stupid, aka Keep it Short and Simple)
5. 'You Aren't Gonna Need It' (YAGNI)
6. Reinventing the wheel (aka Not built here or Not Invented Here)
7. Worse is better (aka New Jersey style)
8. Occam's razor or Ockham's razor
9. Make everything as simple as possible, but not simpler.
10. MoSCoW Method (M - MUST have this, S - SHOULD have this if at all possible, C - COULD have this if it does not affect anything else, W - WON'T have this time but WOULD like in the future.
11. No Silver Bullet
12. The principle of good enough (POGE)
13. The Pareto principle (aka 80-20 rule, or the law of the vital few or the principle of factor scarcity)
14. Satisficing (a portmanteau of "satisfy" and "suffice")
15. Rule of three
16. The single choice principle
17. Greedy reductionism
18. The principle of least astonishment (or surprise)
19. N - version programming (NVP)( aka multiversion programming)
20. Separation of concern

Design Consideration for SaaS Application

1. Asynchronous Logging
2. Asynchronous Auditing
3. Tenant level Logging
4. Tenant level Auditing
5. Tenant Aware Controller
6. Functionality Aware Controller
7. Hierarchy of Meta Data Objects
8. Tenant Aware Data Objects
9. Clear demarcation of data on the basis of where to store – RDBMS, Hierarchical data base – LDAP, File system, NOSQL, etc.
10. Usage of Virtual Database or similar technique
11. Tenant aware load balancer
12. Tenant aware data store
13. Caching to be used at every level – View, controller and data layer and must be tenant aware and policies must be well defined
14. Fault Barrier pattern for Exception Handling
15. Heart beat like mechanism
16. Do not treat business conditions as business exception

Architectural Quality and Consistency: Rules Rule

To maintain architectural quality and consistency at macro (architectural) and micro (implementation) level it is necessary to follow certain rules across the life cycle of any software project and product. From easy enforcement, it is recommended to use automated tool set.

There are three types of rules:

1. Enforced: Must be followed in all cases.
2. Advisable: Should be followed and exception must be documented with reasoning.
3. Guidelines: These are the best practices and recommended to be followed in given context. Exceptions must be documented.

To match with SDLC of any software project, rules are also classified as:

1. Architectural and Design Rules: These rules affect larger part of system and define the resulting product. They form the core of the product.
2. Construction Rules: These rules are applied at micro level, programming aspect of product/application
3. Environmental Rules: These rule define construction environment of product/application
4. Test Rules: These rules affect testing (Quality Control as well as Quality Assurance) at all levels – review of artifacts, unit testing, integration testing, functional testing, user acceptance testing, stress (load) testing, etc.
5. Deployment Rules: These rules affect deployment environment of product/application

For quantitative analysis of architectural quality and consistency, matrices must be identified with respect to each rule. To covey weight of each rule in architectural quality, a weightage should be provided to each metric and the preferable a radar chart should be prepared.

To further enhance the matrices, bugs can be classified as

1. Missing functionality
2. Correctness
a. Absolutely
b. Probably
c. Suggestion
3. Rule Violation
a. Enforced
b. Advisable
c. Guideline
4. Confusing or tricky or non-intuitive

Software Testing and Time machine

Software Testing and Time Machine

What must be in Application or Service to Facilitate Testing

1. Time acceleration and deceleration to test time based calculations
2. Assert like feature
3. Appropriate logging and able to switch logging levels at run or deployment time
4. Facility to run application in Test Mode similar to Training mode
5. Facility to create and manage test scenarios in production environment similar to test users
6. Facility to measure time of execution for various tasks
7. Separation of messages (error, warning or info) and configuration parameters.

Database Archival Utility

1. Introduction

In any enterprise class application physical delete of data is not a choice. There exist only logical delete. But logical delete brings its own challenges. Logical delete of data brings in ever growing tables which keep on slowing the database.
To overcome problem of growing data, in my recent application, I have architected and designed a utility which will move physically deleted data from application tables to archival tables. Archival tables essentially keep deleted records.
This utility is in two parts. Part 1, converts database schema in an XML file where it is marked that which tables to be archived and which are not. Part 2 of the utility does actual movement of data.

2. Database Schema

Database schema can be though of consists of three categories of tables:
• Master-Master Table
• Master Table
• Transaction tables

Master-Master tables are those tables which contain seed data for an application. This data is independent of any user and its operations. Example - Table containing the name of Countries etc.
Master Tables are those tables which are pivotal to an application and hold information like users etc.
Transaction tables are those tables which stores information about all the transaction performed by/on the entities stored in Master tables.

The relationship among tables can be thought of two types:
• Imported: primary keys of other tables referenced
• Exported: tables which refer table1’s primary key

This relationship among tables can grow up to any depth.
To understand please refer following picture:

Table table1 has imported primary key from table2, which in turn has imported keys from table5 and table6 and exported its primary key to table3 and table4. Table table1 also exported its primary key to table7 which in turn has imported primary keys from table8 and table9 and exported its primary key to table10 and table11.

Apart from these relationships, a well designed database schema must not have any circular relationships among tables.

3. Utility Parts

Data archival utility consists of two parts.
• Part 1- Database Relation XML generator Utility: This utility create an XML representation of Database Schema where some one marks each table into said categories (Master-Master, Master and Transaction tables)

• Part 2- Database Archival Utility: This utility is designed for moving, logically deleted records from the Main Database of application to the Archival database of application. DB Archival utility archives records for:
• Master tables
• Transaction tables in Xing

As Master-Master tables contain only static records, its unnecessary burden on utility to move these records to archival database. Instead it is left to the database administrator to copy records from Main Database’s Master-Master tables to Archival database’s Master-Master tables.

4. Database Relation XML generator Utility

Relation XML generator reads a property file to capture credentials to connect to Main database for generating table relations. This property file contains information about the main database. Structure of property files are described at end of Description section of this document.

Relation XML generator reads another property file to identify table for which relation XML is to be generated.

This Utility must be executed after every schema update.

5. Database Archival Utility

DB Archival utility is architected and designed for moving, deleted records from the Main Database to the Archival database. DB Archival utility archives records for:
• Master tables
• Transaction tables

As Master-Master tables contain only static records, its unnecessary burden on utility to move these records to archival database. Instead it is left to the database administrator to copy records from Main Database’s Master-Master tables to Archival database’s Master-Master tables.
DB Archival utility archive records table wise.

Database Archival utility reads a property file to capture credentials to connect to Main database. It also reads another property file to capture Archival database credentials to connect to Archival database.

Database archival utility runs continuously but archives data on specified time only. This time can be specified in a scheduling property file. This file provides information about when to move the deleted records to Archival database.

Database archival utility archives only those deleted records for which the specified timeframe (delay time) has expired since the record was deleted. Currently this timeframe is common for all tables. This timeframe (Delay time) is specified in a property file.

Database Archival utility reads property file to identify tables, which are to be archived. Then for each table, it reads its relationship XML generated by Database Relationship XML Generator, and moves the deleted records to Archival database tables from Main database tables.

Copy and Delete operations

• DB archival utility copies all imported records to archival database tables to avoid any database constraint failure.
• DB archival utility first copies all exported records to archival database tables then delete them from main database tables.

E.g. Lets assume table1 is specified for database archival
.
Exported and Imported tables for table1 are specified in a XML file, created by DB relation XML generator.

DB Archival utility finds all the rows in table1 to be archived i.e. rows in table where status is “Delete” and these deleted records are older than the delay time specified.

DB archival utility reads the XML file for table1 for finding all the tables which are referenced by table1 (Exported Table List) and all tables referenced by table1 (Imported table List).

DB Archival utility copies data of related imported relations to respective archive tables for a particular table (table1).

For exported relation’s DB Archival utility copies to respective archival table and delete the records from main table and all subsequent tables. If any table is imported in exported Table of table1 then its record is only copied to respective archival table.

DB archival utility copy imported table’s data accessed by records to be archived in the table. This procedure is recursive and copies all the data down the hierarchy in the imported table list.

Then DB archival utility copies the exported table’s data to archival table and deletes them from main tables. This procedure is recursive and copies all the data down the hierarchy in the exported table list.

6. Property File Structure

Five property files are used in Database Relation XML generator and DB Archival Utility. Structure of all property files is described below:

• DBCredentials.properties: This property file is read by both utilities to connect to Main database. This file have following keys, all values must be specified and supports only single values.
a. DATABASE_DRIVER
b. DATABASE_IP
c. DATABASE_PORT
d. DATABASE_NAME
e. DATABASE_USER_ID
f. DATABASE_USER_PASSWORD
g. DATABASE_PROTOCOL

• ArchiveDBCredentials.properties: This property file is read by DB Archival utility to connect to Archival database. This file have following keys, all values must be specified and supports only single values.
a. DATABASE_DRIVER
b. DATABASE_IP
c. DATABASE_PORT
d. DATABASE_NAME
e. DATABASE_USER_ID
f. DATABASE_USER_PASSWORD
g. DATABASE_PROTOCOL

• Scheduler.properties: This property file holds information for Scheduler of Archival utility. This file have following keys:
a. SECONDS
b. MINUTES
c. HOURS
d. DAY-OF-MONTH
e. MONTH
f. DAY-OF-WEEK

• TablesToArchive.properties: This property file is read by both utilities to identify Tables to archive. An additional value is read by Database Relation XML generator to identify Master-Master Tables. All other values are read by DB Archival utility. This property file have following keys:

a. TABLE_TO_ARCHIVE

b. MASTER_MASTER_TABLE_LIST

c. Delay time must be specified in property file against appropriate key. By default all keys are set to 0.
 archivalDelayTime_second=0
 archivalDelayTime_minute=0
 archivalDelayTime_hour=0
 archivalDelayTime_day=0

Delay time specifies the time duration after which deleted entity is moved to archival database’s table. Everything must be specified in INTEGER. The least time that can be specified is 0 second.

d. All table names in Main database as key and corresponding table name in archival database as value. All keys and values must be specified in small cases. Although only specifying tables which are in relationship with tables in TABLE_TO_ARCHIVE key will suffice, but to reduce probability of any error provide all tables.

e. Column names of last modified time and current status for all tables to archive must be specified in following manner:
 tableName_LAST_MODIFIED_TIME
 tableName_CURRENT_STATUS

tableName must be provided in small cases. Values for above key must be columns representing LAST_MODIFIED_TIME and CURRENT_STATUS column in respective tables.

7. Assumption in database archival utility

• All relationships in database are maintained at Database level not at code level.
• Archival Database structure will be same as Main database. All relationships will be maintained in archival database too.
• Archival tables must contain one and only one additional column for archival timestamp except for Master-Master table.
• Column names in archival tables must be same as column names in main database table except the archival timestamp column. All other RDBMS components (trigger, stored procedures etc) may not be needed in archival database as it contains only deleted records. If in any case (e.g. reporting etc) any RDBMS components is/are needed they must be modified according to archival Database and ported manually or some other means. Above operation is out of scope for DB Archival utility.
• Tables to archive must be specified in a property file.
• Tables to archive must have a status column and last modification time column. Last modification time column must be stored as timestamp, if it is stored as date, then time part (HH:MM:SS) of date is considered as 00:00:00. Actual column names for status and last modified time must be specified in property file against key
o tableName_LAST_MODIFIED_TIME
o tableName_CURRENT_STATUS
• Archival table names must be specified in property file against there table names as key.
• Deleted records must have there status “Delete” or “delete” or “DELETE” or any possible lower and upper case version of “DELETE”.

Book Review: Successful Service Design for Telecommunications - A comprehensive guide to design and implementation

Book Review: Successful Service Design for Telecommunications - A comprehensive guide to design and implementation: Publisher- Wiley: ISBN- 13: 978-0-470-75393-4

During course of designing a Telecom/ISP product architecture and design product, I encountered Successful Service Design for Telecommunications – A comprehensive guide to design and implementation by Sauming Pang. Before starting the book my impression was that this book will talk about details of Telecom specific service design and implementation. But After reading the book I realized that this book is not confined to Telecom vertical. The principles outlined in this book are universally applicable across industry vertical.

Though book does not mention any of the frameworks but it seems that book is highly influenced by USBOK .

The book is divided into twelve chapters which very clearly depict considerations of design and implementation. The book not only covers technical considerations but also business which are generally overlooked by technical folks.


Disclaimer: I did not get paid to review this book, and I do not stand to gain anything if you buy the book. I have no relationship with the publisher or the author.

Further reading: A complementary book is Telecom Management Crash Course : A Telecom Company Survival Guide (Crash Course) by P.J Louis http://www.amazon.com/Telecom-Management-Crash-Course-Survival/dp/0071386203

One can get more information about book and related topics from:

1. Amazon: http://www.amazon.com/Successful-Service-Design-Telecommunications-implementation/dp/0470753935
2. Publisher -- Wiley http://as.wiley.com/WileyCDA/WileyTitle/productCd-0470753935.html
3. Purchase at Flipkart: http://www.flipkart.com/successful-service-design-telecommunications-sauming/0470753935-0xw3f9w0fq
4. Purchase at A1Books: http://www.a1books.co.in/searchdetail.do?a1Code=booksgoogle&itemCode=0470753935

Template Based Design Technique - Part 2

Now here is a solution of the design problem I posted in Template Based Design Technique.




This diagram not only covers the flow explained in Template Based Design Technique but also CRUD operations on Resume and Resume Template. This solution assumes God’s list is not updatable via any user interface, you need developer to update God’s list.

Role Based Authentication & Authorization

Today’s most of the applications have role based authentication and authorization systems. On the surface it seems that it is very simple and straightforward but as we dive into details, various design challenges emerges.

In simplistic terms role based authentication and authorization can be depicted as



User 1 is assigned to Role 1
User 2 is assigned to Role 1, Role 2 and Role n
User 3 is not assigned to any Role
User p is not assigned to any Role

Role 1 has Permission 1 and Permission 2
Role 2 has Permission 2
Role 3 has no Permission
Role n has no Permission

Systems has Permission 1, 2, 3, …, m

As we go into details of Figure 1, one easily deduces that system can have orphan users, roles and permissions, which may not be feasible state for a system.

While designing a role based authentication and authorization architect and designers have numerous options and some of the options may be conflicting.

Option Type: Orphan status
Orphan Users -- Allowed/Not Allowed
Roles -- Allowed/Not Allowed
Orphan Permissions -- Allowed/Not Allowed

Option Type: Permission Type
Permission -- Only Positive/Only Negative/Both

Number of Roles to a User
One Role per User
Multiple Roles per User

Book Review: Practical API Design: Confessions of a Java Framework Architect

Book Review: Practical API Design: Confessions of a Java Framework Architect by Jaroslav Tulach: Publisher- Apress: ISBN- 13: 978-1-4302-0973-7

Before start reading Practical API Design, I googled and not able to find much of stuff on API design. Only worth while to mention are discussion on Artima forum dated 2005 and a paper by Brian Ellis, Jeffrey Stylos, Brad Myers. Certainly Effective Java Joshua Bloch also discusses the topic. I was surprised and started reading the book.

Once I started the book I was hooked to it. This book is written clearly demarcated three parts: Theory, Design aspects and Advice characterized as Theory & Justification, Practical Design and daily Life.

As confessed by Tulach book covers only Java language specifics but still book is must read for non java professionals.

Book primarily focuses on few topics: Evolve-ability, Backward & forward compatibility, architecture by committee and lessons learnt from NetBeans API development.

The book is serious readers and those who like to read book with notebook and pencil. This book is not easy read which you can read during your lunch. It needs serious time and efforts to read and then grasps. It discusses various aspects of API design with respect to backward & forward compatibility, runtime behavior, testability, separation of API & SPI, mingling with other APIs (here certainly I recommend reads to look into Struts 2.x and Grails architecture, design and source, because these frameworks/API uses third party libraries extensively), interface vs. abstract classes etc.

Tulach has recommended not to rewrite because of breaking old loyal clients. Personally, I also subscribed to this line of thought but certainly Struts 2.x violets this line and still very successful. It has to be studied in details.

Though book is comprehensive but it has not covered few topics:

1. Effect of hard coding of SUID (static final long serialVersionUID =xx)
2. Performance
3. How to make API accessible from different programming languages (say C#)

Tulach is maintaining books web presence at http://www.apidesign.org, I hope he will take care of the different views and uncharted topics in future.

I am hoping for similar book by other veterans of industry both open source & proprietary frameworks and APIs (Eclipse, Spring, Hibernate, jUnit, Glassfish, Ant, Struts, Groovy on the Grails) as well as non Java frameworks and APIs (Ruby on the Rails , AspectDNG, Netron graph library, NAnt).

Practical API Design must be in your bookshelf if you are java architects veterans or aspiring one. It is also a good reference book for non java folks as principles discussed are very much applicable in other languages but must be tweaked to adjust with language specifics.

Disclaimer: I did not get paid to review this book, and I do not stand to gain anything if you buy the book. I have no relationship with the publisher or the author.

Further reading: A competing book is Effective Java by Joshua Bloch

One can get more information about book and related topics from:

1. Book’s web presence at http://apidesign.org
2. Purchase book at Amazon
3. Publisher -- Apress
4. Interview with Author
5. Review at 72 Miles
6. 90th Percentile Blog
7. The Factory Pattern in API Design: A Usability Evaluation by Brian Ellis, Jeffrey Stylos, Brad Myers
8. Discussion on API Design
9. Java Language Specification, Chapter 13: Binary Compatibility