In a world full of information, organizations feed on data to make all kinds of decisions. Your company’s database graph (also known as Schema) is a key document that helps you structure your information. Whether operational or strategic resolutions, decisions on logistics, marketing, or finance, data takes a crucial role in this process. The awareness and use of customer data management systems, management information systems, and big data is growing. But data management is not an easy task.
Data is stored and manipulated in databases. We can understand that databases, then, represent a fundamental pillar for companies. A correctly-designed database graph offers customers admission to crucial information. By following the standards and best practices we’ll discuss in this article, you’ll be able to layout a database that works nicely and fits your organization’s needs.
What are the components of databases?
Databases are made up of different elements that give life to the interactions between data. We can identify three major components of a database:
1. On the one hand, tables represent a set of homogeneous data with a defined structure. Data is organized into fields (also known as columns) and records (also called rows). Having different tables within a database can be very useful since its correct administration helps to avoid data redundancy and to optimize processes.
As the second fundamental pillar of database management, we must mention the relationships between tables. These relationships are links to one or more tables from a field that they share. This field, generally called a key, can represent an identification that allows recognizing the specific record that each row represents. In this way, it is possible to combine different tables to take advantage of their interrelationships.
Last but not least, we have the normalization of databases. It is a necessary process for a database to be used optimally. Thanks to it, we can focus on avoiding data redundancy and guaranteeing their referential integrity. Normalization is the process that allows one table to communicate with another and for the data and information type to be compatible. It can also help us to interconnect different databases to take advantage of their joint management.
The interactions between the tables occur thanks to relational algebra. These are all the logical and mathematical operations that work at the back end of a database management system. It allows us to create a relationship between tables, allowing us to retrieve data efficiently.
Relational Schema: how to design a database graph.
A relational Schema is a database graph, or a graphical representation that allows a data architect to have a reliable idea of how the database must be organized. It is a super useful graphic in the process of planning the structure of a database. It is made up of the tables, the interactions between them, and the different keys that allow them to be used together.
To build a relational schema, we must indicate the following elements on the graph:
Each table, represented as an individual rectangle. For example, a company might have one table with product information, one with customer information, one with production costs, one with sales costs, and one with sales information. These five tables could interact in a database.
Each column of each table, which will be a line within each square. For example, in a table with product information, we might have the following columns: product ID, product name, product brand, product type, product color, and product size.
The primary keys. This is a column (or set of columns) whose value exists and is unique for every record in a table. Surely, reading the concept of the primary key, you can imagine that a common example is the ID of a product or a customer.
The foreign keys. These are columns that identify the relationship between tables. Generally, the primary key of one table is used as the foreign key of another. For example, a table with sales information can use the product ID column as a foreign key to look up product information in another table.
The relationships between the columns, that tell you how much of the data from a foreign key field can be seen in the primary key column of the table that the data is related to.
The types of relationships:
One-to-many: one value from a column under a certain table can be found many times in a column from another table.
One-to-one: each unique value of a column of a table can only appear once in a certain column of another table.
Many-to-many: there is no restriction on the number of times the values can be repeated.
It seems very complicated, but with a little practice, a developer becomes adept at creating databases using this tool. At Huenei we always structure the databases of our projects using relational schemas, since they help us to enhance the operation of our software products.
As you can see, the correct management of these databases could allow a company to reduce the redundancy and inconsistency of data, reduce the difficulty for interested parties to access them, avoiding data isolation. Additionally, database administration focuses on correcting anomalies in concurrent access, reducing security problems, and also data integrity and consistency.
Relational schemas help software factories achieve smoother and more efficient development!
Understand What Big Data is and How It can Help your Business Strategy
We are living in the age of information revolution and Big Data. Huge volumes of data are generated and used by businesses and consumers every day. We use them to make decisions in our daily lives, to work, to inform ourselves… And this is the case in all sectors: retail companies, health, security, entertainment, government, science, and so on.
An important opportunity arises here for companies like yours. Being able to manage, analyze and interpret the information and knowledge available can allow your organization to get closer to your customers like never before and achieve understanding of them, supporting operations with better management of the internal and external processes of your company.
What is Big Data?
It is a complex and high-volume dataset that, given its characteristics, traditional tools are unable to process in a reasonable time and cost range. We are talking about datasets that exceed the capacity of common software to be captured, managed and processed. This concept is becoming the heart of the Data Science strategy of many companies. For its management, we need to use special technologies that allow us to work with huge volumes of structured, unstructured or semi-structured natures, which can come from sensors, microphones, cameras, scanners, images, social networks and the internet, among many other sources.
By implementing Big Data projects we seek to better understand our business information and the problems we need to address, explain relevant events, and create models that allow us to predict future events and behaviors. We help ourselves with the use of technologies and resources that facilitate the exploration and analysis of these enormous volumes of complex and varied inputs. The objective is clear: convert data into information that facilitates decision-making in real time.
Data has become critical across all industries and business functions. Regardless of the field or sector in which your company works, a new datum is born every second, which can represent great opportunities. If you are not taking advantage of them and implementing them in the core of your business strategy, you are wasting information and knowledge that are key to success.
Big Data creates value in different ways for your organization. It is a factor of competitiveness and leads to increase productivity and value for the consumer, the organization and society in general.
We already understand the concept and its importance in companies like yours or Huenei. Now we want to tell you about the fundamental characteristics of Big Data. These are magnitudes that define it and that have been studied from different analysis models. The original approach talks about the 3Vs:
Volume. Inputs of facts, information, evidence, etc. can be internal or external. This causes millions of GB of data and information to be generated, stored, managed, analyzed and used every minute.
Speed. Information is continuously being generated, so companies need to be able to access, store, analyze and use it immediately in order to make decisions.
Variety. Data is not homogeneous. That is, a wide variety of structured, unstructured, and semi-structured natures take part of the analysis. We will talk more in detail about them in another article later!
Over the years, other models have emerged that speak of the 5Vs, 7Vs and even 8Vs…
Illustration by Sphinx Worldbiz Limited
Big Data Analytics
At Huenei, we firmly believe that database preparation takes about 80% of an analyst’s time. Database preparation covers all activities to build the final data set from the initial raw data. This is an important process because, due to the 3Vs (or the number of Vs you want to imagine), it implies that the raw data is usually corrupt, that is, it is noisy, incomplete, has missing attributes, outliers, missing values, lack of structure, and other characteristics that make analysis difficult. The preparation stage solves these types of problems to ensure that the final data set used for analysis and modeling is correctly structured.
Then we can move on to the fun instance: Analytics. It is the process of sifting through large volumes of data to discover hidden patterns, unknown correlations, and other useful information (such as market trends or customer preferences) that can be used to make better business decisions.
There are different tools for Data Analysis. Some can be used for free and others are for commercial use, so they have a cost in the license of use.
We hope that after talking about the characteristics of Big Data and its importance for organizations like yours, you have been able to understand the value it offers for the structure and strategy of your business. The process of implementing the use of enormous amounts of data can take time and involve large software and data engineering developments that allow supporting large constant information flows. At Huenei we have experience working with software development projects of all kinds in a wide variety of industries: entertainment, financial, government, retail, and more. We love helping our clients implement projects of all natures, such as the implementation of this kinds of projects in the core of their businesses.
Data is a vital resource for any organization. Managing business data requires a careful and standardized process of different kinds of data structure. We have already discussed in previous articles the life cycle of data and how it can help your company in making business decisions. This is why today we propose to take another step into the world of data and understand what types of data companies like yours work with.
Database management problems are often related to tight behaviors in the organization. That is to say, inconveniences with the treatment of the data that arise from the use of outdated, inefficient technologies that consume many organizational resources. This translates into a high dependency between the programs used and the data, little flexibility in administration, difficulty in sharing data between applications or users, data redundancy, and poor information security.
But even in advanced technology companies, it is common to find the same limitation: staff does not understand the types of data they are working with and have difficulty transforming the data into key knowledge relevant for decision making. And with the advancement of Big Data in companies, these problems represent a loss of value for customers, employees, and stakeholders.
Data in companies: different data structures.
Everyday companies collect (and generate) a lot of data and information. With the advancement of technology, data that lacks a defined structure became accessible and of great use for making business decisions; years ago, it was almost impossible to analyze these data in a standardized and quantitative way. Let’s see what the alternatives we face are:
Structured data. They are traditional data, capable of being stored in tables made up of rows and columns. They are located in a fixed field of a specific record or file. The most common examples are spreadsheets and traditional databases (for example, databases of students, employees, customers, financial, logistics…).
Semi-structured data. These do not follow a fixed and explicit scheme. They are not limited to certain fields, but they do maintain markers to separate items. Tags and other markers are used to identify some of its elements, but they do not have a rigid structure. We can mention XML and HTML documents, and data obtained from sensors as examples. Some other not-so-traditional examples that we could mention are the author of a Facebook post, the length of a song, the recipient of an email, and so on.
Unstructured data. They are presented in formats that cannot be easily manipulated by relational databases. These are usually stored in data lakes, given their characteristics. Any type of unstructured text content represents a classic example (Word, PowerPoint, PDF files, etc.). Most multimedia documents (audio, voice, video, photographs) and the content of social media posts, emails, and so forth, also fall into this category.
How do I structure my data?
Beyond the level of structure discussed above, it is essential to your organization’s data management process that you can standardize its treatment and storage. For that, a fundamental concept is that of metadata: data about data. It sounds like a play on words, but we mean information about where data is used and stored, the data sources, what changes are made to the data, and how one piece of data refers to other information. To structure a database we have to consider four essential components: the character, the field, the record, and the file. So we can understand how our data is configured …
A character is the most basic element of logical data. These are alphabetic, numeric, or other-type symbols that make up our data. For example, the name PAUL consists of four characters: P, A, U, L.
The field is the grouping of characters that represents an attribute of some entity (for example, data obtained from a survey, from a customer data management system, or an ERP). Continuing with the previous example, the name PAUL would represent a complete field.
The record is a grouping of fields. Represents a set of attributes that describe an entity. For example, in a survey, all responses from Paul (a participant) represent one record (also known in some cases as a “row”).
Last but not least, a file is a group of related records. If we continue with Paul’s example, we could say that the survey data matrix is an example file (whether it is encoded in Excel, SQL, CSV, or whatever format it is). Files can be classified based on certain considerations. Let’s see some of them:
The application for which they are used (payroll, customer bases, inventories …).
The type of data they include (documents, images, multimedia …).
Its permanence (monthly files, annual sets …).
Its possibility of modification (updateable files –dynamic, modifiable-, historical –means of consultation, not modifiable).
As you have seen, the world of data is exciting and you can always continue learning concepts and strategies to take advantage of its value in your organization. To close this article and as a conclusion and example of the value of data for companies, we want to invite you to learn about a project in which we work for one of our clients. The General Service Survey that we develop for Aeropuertos Argentinos is an application of the entire life cycle of data (from its creation to its use) and is fed with data of different levels of structure. It is about the development of a platform to carry out surveys to visitors and employees, together with the analysis and preparation of automated reports. Don’t miss this case study!
Data is present on the everyday of organizations like yours and data analysis skills are key for business success. Data helps us learn from the past and make better business decisions in the future. The advancement of data science in the business worls makes us understand the importance of a correct management of the life cycle of your company’s data.
We are now in the age of the data revolution. Increasingly, organizations have greater volumes of data that allow them to optimize their decisions. Big Data, as one of the foundations of many companies, represents a challenge but also an opportunity for organizations.
The opportunity lies in being able to satisfy consumer needs in a more effective way, understanding their wishes but also optimizing operations and reducing associated costs.
On the other hand, the challenge is related to the necessary infrastructure for this type of processes, as well as being able to count on trained personnel to carry them out.
It is therefore essential that companies have collaborators who are capable of analyzing, managing, reporting, and adding value to the data. The digital economy, digital transformation and globalization accelerate this process rapidly.
Professionals in the world of data science, business intelligence and data analytics generally have high levels of education and training. A very solid educational background is generally necessary to develop the depth of knowledge required to be a data or business analyst. The reality is that most of the professionals in this field are in constant training. Data analysis skills are key for success.
What Are the Data Analysis Skills You Need?
We cannot give a single answer to this question. Depending on the profile of the professional and the tasks demanded by the position, data and business analysts will need to have different types of skills. We have developed a list of what we think are the most important skills to consider. This list is not necesarilly comprehensive, and it is not a requirement to be an expert in all these areas to be able to develop in the world of data science and business analytics. However, we have developed this list based on the work we have done for our clients. Let’s explore it!
Spreadsheets like Excel and Google Sheets. They allow handling mainly small volumes of data, focusing on the use of descriptive statistics, pivot tables, formulas, and simple visualizations. It is an interesting tool for developing a BI profile, but it could be somewhat limited for more comprehensive Big Data processes.
Visualization. The results of the data analysis are finally presented to other teams and to other people and users. In this sense, visualization skills are key for any profile. We need to make sure that we are choosing the correct type of graph for the data we are visualizing and that the visualization and narrative is done in a clear and easy-to-understand way. Some tools that business analysts use are Tableau, Power BI, and Google Data Studio. However, companies focused on data and information often develop ad hoc and tailored dashboards and monitoring platforms, adapted 100% to the specific needs of each project. You can find out about the work we performed for Aeropuertos Argentinos, where we developed a platform for surveys, automatic data analysis and reporting of results.
Dashboard example by Power BI
Statistics. Statistical skills are key for any data analyst. From basic descriptive statistics, to inferential statistical analysis and more advanced models of Machine Learning (e.g. Artificial Neural Networks) can help your organization to enhance its data strategy.
SQL – Structured Query Language. It is a programming language that can help you perform operations such as adding, removing, and extracting data from a database. It can also help you perform analytical functions and transform database structures. Database managers must be familiar with databases and languages for their administration, as well as with environments that allow the management of databases (some widely used environments are: Oracle, SQL Server, MySQL, MongoDB).
It is important that you be able to master SQL if you want to be a data scientist. SQL is specifically designed to help you access, communicate, and work with data. It contains concise commands that can be useful to save time.
Programming languages. Mastering certain programming languages can make data analysis much more flexible. The languages we would like to highlight are R and Python.
Python is one of the most popular programming languages, along with other highly acclaimed languages such as Java, Perl, or C/C ++. Python is a great programming language for scientists and data analysts. Due to its versatility, you can use Python in almost every step of your business operations.
As you can see, you need various knowledge to develop in the world of data science, business intelligence and data analytics. As we have mentioned, your organization may need a particular profile that requires different knowledge than those presented above. In order to adapt to the needs of each project, companies like Huenei offer you outsourcing solutions for specific software developments.
You already know that at Huenei we are true defenders of the value of information, knowledge and competitiveness in business. We have previously discussed that data represents one of the most important assets of any company today. Correct management of the data lifecycle is key to achieving levels of knowledge and competitiveness that enhance the strategy and competitive advantage of your organization.
At Huenei we focus every day on helping our clients achieve their strategic business objectives from the development of software and information systems that allow them an efficient use of organizational resources: data and information, economic or financial resources, intellectual capital, physical resources, logistics, among others.
In this sense, it is imperative to have systems that allow us to transform data, obtained from various sources, into relevant information for decision-making. Consequently, the organization will be able to acquire the knowledge necessary for making informed decisions.
But… How do we achieve competitiveness? In this article we are going to focus on understanding how raw data goes through a series of steps until it gets to represent a fundamental part of organizational strategy. This journey carried out by the data is transforming and adapting it based on issues such as organizational needs and objectives, the context, previous experience… We propose to structure this exploration with a pyramid that we like to call the Pyramid of Competitiveness.
The definition of business competitiveness depends on the special treatment of data, which allows it to be transformed into different instances of knowledge, managing to lead to an underlying differentiating strategic capacity that gives an organization a certain advantage over its competition. This approach is based on theoretical foundations that different authors offer about knowledge management in organizations, but it is also evidenced by specific case studies on real companies that take advantage of data and information for strategic development. (We’ll look at some examples of Huenei customers below!)
Let’s analyze together each of the components of the pyramid of competitiveness!
A datum is a symbolic representation of an attribute or a variable. It is a record of interactions with customers, users or interested parties that is represented as a discrete element, a fact or objective record of a certain event. It can arise from various sources internal or external to the organization. The main characteristic of the data is that it is a representation of entities or facts, out of context. A datum itself is not feasible to generate value for the organization. Some common examples of data are dates, amounts, the address of an individual, a specific symbol or character, the tracking of logistics, financial activities, etc.
Information is generated as an organized set of processed and analyzed data, with some specific meaning for the user. This is data that has been analyzed, reduced and organized into patterns.
Data is the raw material from which information is produced, where it is collected, ordered and reformulated in order to make it known and used. In this sense, information can be understood as the context and meaning assigned to data. The classic examples of information are the results of a market research, a sales report, a management report of distribution and logistics, the financial statements of the organization.
Knowledge, which some also call “findings,” can be understood as information that allows creating value, through its understanding and combination with previous experience. It is the information “distilled” by the human intellect. The information is then adapted to the context and the specific objectives and needs of a company.
In general we try to structure, close or save knowledge in documents, databases, processes, etc. to be able to take advantage of it in the future. The definition of a market segment, the development of a sales strategy, logistics planning and budget allocation of a company can be understood as common examples of organizational knowledge, that is, how we make intelligent decisions in the company.
We come to the key instance of this pyramid. Business competitiveness is the ability of a company like yours to generate valuable offers for customers, which guarantees good profitability and allows sustainability over time. The knowledge added to the accumulation of significant experiences of the company give rise to competitiveness.
This stage is reached after a long road and as a result of data processing and the correct understanding of the information, so that it becomes knowledge, and by repeating this cycle, a company reaches a position of wisdom that affects business decisions positively and from an insightful perspective. The example that we can provide is only one: the competitive advantage of the organization. This can take different forms and rely on different capacities, but companies with a real competitive advantage are those that have reached a position of sustainable competitiveness.
Competitiveness Case Studies: Huenei Clients
We would like to end this article with some practical examples of what we have been talking about. Let us quickly mention some cases where we have helped our clients to enhance business competitiveness through software developments. In all cases, you will be able to learn more by analyzing the case studies that we have prepared for you.
Management Portal for ABB. We developed a collaborative portal so that ABB members can manage and distribute information between different business areas.
High Frequency Claims for ACE Insurance. We collaborated with the realization of tests on an application to improve the quality of service by taking advantage of the company’s information on clients and transactions.
Material Request for YPF. We developed a mobile application aimed at allowing employees to make requests for materials, optimizing the flow of information on orders and speeding up operational competitiveness.
General Service Survey for Aeropuertos Argentinos. In this case, we develop a System to collect data from travelers through surveys. The System also allowed an automated analysis of the data and the generation of reports (information) for decision-making.
Now that you know the Pyramid of Competitiveness, we invite you to take a reflective role on how data goes through this process in your organization. Your objective should be to understand whether or not the data and information available to you in your organization are capable of being used from the point of view of the knowledge and competitiveness that they could develop.
As we have discussed in previous articles, we currently live in the era of data revolution. This represented a paradigm shift a few years ago for organizations that had to stop making gut-feeling decisions and move towards decision-making based on data, information, and knowledge. This pyramid of competitiveness of organizations requires exhaustive management of data and database relationships. Also, a dynamic understanding of the needs of stakeholders, the company, and consumers is key to succeed.
At this point, the importance of customer database administration arises. The management of software systems is a fundamental pillar of an organization. Data-centered activities need to have one thing in common: correctly managed database relationships. A database is understood as a set of data that is organized for a specific use, related to a particular matter or purpose.
To illustrate its importance, we’d like to let you know that at Huenei we work with large databases with information from various areas: customers, suppliers, partners, financial information, and logistics, among others. The interrelationships between various tables in our database allow us to efficiently manage our business. Additionally, we have worked on many database-powered software systems implementation projects for our clients. We’ll tell you about some concrete examples later.
Main types of database relationships for your business.
We can find different natures of databases in terms of their structure, their operation, and feeding. However, a large majority of them can be compressed into two broad categories that can be the key to managing your business databases.
On-Line Transactional Processing (OLTP)
They are transactional databases, designed to load and access your data quickly. Transactional means that they are used for the rapid sending and receiving of your organization’s data. A typical example could be when you send money to a provider’s bank account to make a payment.
Surely you have heard of some of the main OLTP products. But did you already know they are OLTP? Let’s review some of them…
Enterprise Resource Planning (ERP). It is an integrated information system that covers all the business functions of a company: purchases, payments, provisioning, stock… They pursue the objective of optimizing business processes, allowing access to information in a reliable and timely manner, and facilitating the possibility of sharing information among all participants in a business process. We have developed different ERP products, such as the work we carried out for ABB, for whom we work in the development of a portal to manage and distribute information among different business areas.
They are analytical databases, which allow the reading of large amounts of data to get to extract some type of useful information, such as sales trends, behavior patterns, waste analysis, complex logistics reports, etc. Their objective is to speed up the query of large amounts of data and extract useful information. This is achieved by dinamically managing database relationships.
The master and exemplary case of OLAP is the famous Business Intelligence (BI). We have already talked about the importance of BI in strategic management, but from a database and software perspective, we understand it as computer applications that interpret historical data, analyze trends and measure performance aimed at serving as support to decision-making. An example that we would like to tell you is our recent work done for Edenor. We developed a software product that helped them optimize the company’s data analysis system, seeking to achieve a daily curve of the demanded energy by area.
OLTP vs. OLAP:
Facilitate fast data insertion.
Facilitate high-performance queries at high speeds.
Store transactions at a high level of detail.
Consolidate data in an automatic and optimal way.
Current or recent data.
Areas may not be integrated.
Integration of the entire organization.
Does your company already use OLTP and OLAP databases? Now you know that these can bring great benefits to your organization. We invite you to read some of our case studies so that you can see more concrete examples of projects that we have carried out for our clients. At Huenei we have extensive experience in the development of software products for data management and analysis and we would like to be able to help your organization achieve its business objectives using data.
Get directly to your mail the latest trends and news in Software Development, Mobile Development, UX / UI Design and Infrastructure Services, as well as in the management of Dedicated Teams and Turnkey Projects remotely.
Subscribe to our mail and start receibing all of our information.