DoE and multivariate statistical approaches have been identified before as a clear way of saving time and resources (1). They are systematic and structured approaches to product development and process improvement. The methodology is based on introducing variability into the system by changing a limited number of variables at controlled levels simultaneously, but systematically, in order to study the parameter space. The aim of a DoE is to maximise the knowledge obtained while minimising experimentation. It can also help to ‘fail quickly’; if, for example, the outcome of the study is that the variability cannot be explained by changes on any of the variables, further variables need to be considered. This leads again to saving time and resources.

Statistical modelling of designed or undesigned data can provide a predictive model. This model can be used to predict the output from a combination of the inputs that has not been tried before experimentally, providing that the combination is within the experimental space. Therefore, the predictive aspect of the model can potentially save unnecessary experimentation in the future.

Although it is difficult to quantify and compare like-for-like, some studies in the pharmaceutical space stated that projects involving multivariate experimentation resulted in a requirement for 50–70% fewer batches than traditional experimentation, such as a ‘one factor at a time’ (OFAT) approach. Therefore, the total number of product development weeks were reduced by at least 43% (1), illustrating that time can be saved using this approach.

Historically, multivariate experimentation has not been very accessible for non-statisticians, but now it is possible thanks to user-friendly statistical software packages like JMP^® (from SAS Institute, USA), which offers extensive DoE capabilities to design and analyse all types of DoE, and visualisation tools which allow the user to understand which experiments are being carried out and to visualise and communicate the results effectively. Despite potentially remarkable savings, the implementation cost should be relatively low, since it only requires making a software package like JMP^® available to scientists and having a good network of support and coaching internally within organisations to share good practices and new methods.

Although cost reduction is possibly the strongest advantage of using statistical tools, it is not the only one. The structured approach embedded in statistical experimentation allows a better project planning process with known schedules.

When planning an experimental programme, the use of DoE methodology provides several advantages over a traditional approach. The scientists involved in the work must first identify all the variables, thinking about the entire system, which helps to ensure that the project scope is properly assessed and clearly defined at the outset. The variables should be split into those which can be changed (factors or inputs) and those which are to be affected because of these changes and measured (responses or outputs). Are the factors continuous or categorical in data type? Can the factors be controlled during the experiments? If they cannot be controlled then should they be measured? Which of the factors will be fixed as part of the scope of the experimental programme and which will be varied? What are the ranges of each factor? How many levels for each factor will be used? These questions are best answered by a team of scientists with pre-existing knowledge and skills. The planning stages of the DoE are crucial to its outcome and should not be overlooked.

The DoE methodology of variable identification facilitates project definition and considers the experimental design space in its entirety before focusing on the parts of interest. Experimental design space is the total space defined by the factor ranges. This must be carefully chosen by the scientist to ensure that the aims of the experiment can be achieved. An illustration of the design space for a three-factor experimental design is shown in Figure 1, where the design space is the three-dimensional area within the cube, and experiments can take the form of any combination of the three factors within this design space. This can often be neglected during the planning of non-DoE type experimental work.

Fig. 1.

The experimental matrix generated by the DoE is also important in project scheduling. Access to the full set of planned experiments at the start of the project helps when assigning resources and provides a good estimate to management about exactly how long the programme will take. It also prevents interim interpretation of the data because the full set of results is necessary for analysis. This is in direct contrast to typical ‘reactive’ laboratory practice whereby each experiment is analysed immediately afterwards and used to inform the next experiment. In this traditional way, the end of the programme is not clear as the total number of experiments has not been defined and is therefore likely to take longer. DoE is a more proactive approach with a clear timeline for project management purposes and ensures that the full dataset is available before analysis, decreasing the chances of drawing incorrect conclusions or subjectively changing the parameters of the project based on the results of the latest experiment.

An example of this proactive approach coupled with demand for a tight project schedule was demonstrated within Johnson Matthey. An online analyser was loaned from an instrument manufacturer to investigate whether it could be used to monitor a chemical reaction in real-time on plant. The analyser was only available for two weeks and it was therefore important to study the effectiveness of the analyser as efficiently as possible, by collecting spectral data accounting for a range of reaction product mixtures. The aim was to provide enough variation to ensure that robust calibrations for each component in the product mixture could be established within the range of expected online process conditions. A screening DoE was generated to assess the influence of six factors on the spectral response for each reaction product mixture. Two centre points, set in the middle of the factor ranges, were included to determine whether non-linear relationships between the factors and the spectral response could be present, and to establish whether the spectral response was repeatable. The DoE generated 17 experiments, which were run in a randomised order (Table I). These 17 experiments were combinations of high- and low-level settings for each of the six factors, ensuring that there were no correlations between each of the two factors and that effects on the response can be independently quantified (Figure 2).

Fig. 2.

Excellent calibrations for all components of the reaction product mixture were obtained, and the instrument manufacturer commented on how well the design space had been explored in the time available using the DoE. Following successful demonstration that this online analyser could be used to monitor the reaction in all expected conditions, proposals were submitted recommending its purchase and operation on a customer plant. The advantage offered by statistical tools to draw trustworthy conclusions is an aspect which deserves proper consideration.

Trustworthy conclusions obtained from a study and its data are necessary to make the right decisions. The conclusions obtained from the data are as good as the data itself. Therefore, quality of the data is a key aspect. For example, if the data is biased or unbalanced, there is a possibility of obtaining inaccurate conclusions which could lead to unsuccessful or suboptimal decisions for the system or process. The use of statistical tools to plan the study should ensure good quality data and therefore increase the probability of drawing reliable conclusions.

‘Universal versus local optimum’ is an issue which can occur if the data to analyse is not a good representation of the experimental space being studied. In that case, the data analysis can lead to a local optimum of conditions to maximise the output, while the universal optimum is still to be discovered (Figure 3). Following traditional experimentation only data in the red path was obtained, leading the scientists to reach a local optimum. However, within the experimental space defined, a better outcome is possible, but has not been found. This is what it is referred to here as the universal optimum for the experimental space.

Fig. 3.

DoE leads to obtaining the right data since the experiments are designed to study the effect of the selected variables and understand the system or process in the most efficient way. It ensures the data is balanced and distributed within the experimental space, allowing unbiased and relevant conclusions about the system to be extracted. JMP^® software is a leader in statistical DoE, making multiple state-of-the-art designs available for scientists to choose from depending on the specific case.

When dealing with historical data, which could be biased, for example rich in certain areas of the experimental space and sparse in others, the risk to find a local optimum instead of the universal optimum is significant. Working with historical data can not only lead to suboptimal decisions but can also be time consuming, so the use of statistical tools within JMP^® can significantly support this process.

The use of advanced data analytics may be applied effectively to existing datasets. There are many instances in research and development (R&D) and manufacturing where large datasets have been generated from previous work programmes which could prove useful as a starting point for the current project of interest. Rather than starting completely from scratch, it may be possible to identify trends and relationships between variables from this existing data. This has the advantage of utilising historical data, much of which was probably expensive and resource-intensive to generate. The use of exploratory data analysis tools within JMP^® facilitates this process.

An example of analysing historical data with an exploratory approach has been demonstrated within Johnson Matthey at a catalyst manufacturing site. Two separate plants were involved successively in the production of a single catalyst product, and the multivariate tools within JMP^® were used to determine which of the process inputs most affected the properties of the intermediate material (output of Plant 1), and then which of these as inputs affected the properties of the finished catalyst product (output of Plant 2). The process data used in this analysis was taken from several years of production on both plants. An example of part of the exploratory data analysis used for this example is shown in Figure 4, where the distribution and graph builder platforms of JMP^® were used to visualise relationships between variables. Based on the results of the analysis and the predictive models created, process settings were changed to optimise catalyst product properties and both plants now have higher rates of meeting target specifications.

Fig. 4.

Limitations may exist in the historical data, and probably will be present if the data was collected using a traditional OFAT approach rather than from a designed set of experiments. In this case, it is important to identify where multicollinearity exists and how this affects the analysis of the dataset and the conclusions drawn. The multivariate and exploratory tools within JMP^® allow these limitations to be visualised and understood, enabling the scientist to make informed decisions about what the data is showing while being mindful of the underlying assumptions. It also provides an opportunity for sequential experimentation, whereby the existing data, although limited, can be used as a starting point for a subsequent DoE which can deconvolute the limitations in the historical data, resolving the correlated effects and suggesting the best combination of experiments in parts of the design space with fewer existing data points. Alternatively, the understanding gained from mining the historical dataset may be used to focus on fewer significant effects for a new experimental design with additional factors. Related to predictive models, numerous advantages can be drawn for the prediction capabilities, such as gaining control and adaptability.

An important advantage of the predictive capacity of a model is the control over the system or process that it offers. It allows the scientist to respond to the outputs and modify the inputs in a system or process to adapt to a new situation, keeping the system or process on target. For example, if the value of one of the input variables changes for external reasons out of our control, the model will point out what the value of the other inputs should be which can be controlled to keep the output on target, compensating for the changes in the input without any experimentation needed. This brings control back to the users and offers tremendous flexibility and adaptability; very important qualities in the fast-moving world. This is often used within Johnson Matthey in different businesses, for example, formulations for certain products to ensure the quality of the final or intermediate product, by proactively adapting to changes in the raw materials.

This task is performed easily in JMP^® using the interactive ‘prediction profiler’ (Figure 5). The profiler also allows the scientist to find a new optimum combination of input values if the output target changes (for example, a new customer specification), or when an input needs to be fixed at a certain value (for example, a new requirement or limitation). The profiler will find the optimal combination of the remaining input variables to stay on target.

Fig. 5.

Control over systems and processes is not the only advantage of data modelling. Another very important aspect is related to knowledge storage.

In a scientific process, data is generated to obtain answers to technical questions, prove and contradict hypotheses and corroborate assumptions in the process of discovery or optimisation. Therefore, the data itself is a vehicle to get knowledge. Knowledge is the final aim, but that knowledge ideally needs to be recordable, communicable and transferable to maximise its use.

Statistical modelling allows knowledge to be extracted from a study or from data in the shape of a model that helps to communicate and visualise the effects of the different inputs on the output. The model itself contains this knowledge and allows the rest of the world to utilise that knowledge.

Within JMP^®, statistical modelling is accessible to everyone with multiple modelling techniques available and the ability to compare them easily. In addition, the software offers the prediction profiler tool (Figure 5), which not only enables scientists to visualise and communicate their findings (contained in a model) dynamically and interactively, but also to transfer and share the learnings with colleagues in the same team and between different teams and functions. Utilising these tools can ensure that the knowledge obtained from experimentation stays in the company in a reusable format despite employees leaving or retiring.

Another aspect to facilitate knowledge sharing comes from the understanding of a chemical problem or question. Sometimes this can be very subjective and variable depending on the scientist’s background, expertise and interests. JMP^® tools offer enhanced visualisations for different stages in the process to ensure good communication and visualisation of problems and results.

Visualisation tools are used in different steps of data analysis and are key to helping understand and communicate the chemical problem studied. In the first instance, they are used to explore the data set initially. This process is very important as it helps the scientist to get to know the data. On one hand, it helps to understand the experimental space and identify possible gaps, outliers and errors. As mentioned before, this stage is particularly important when looking at historical data as this data tends to be limited. It can also help the scientist to identify correlations between the inputs and the outputs before embarking into model building. The ‘graph builder’ and ‘distribution’ platforms available in JMP^® are excellent tools to use at this stage (Figure 4). They are also great tools to present a point or argument in a meeting since they are interactive and easy to understand. All these visualisations can also be shared using dashboards that can be produced in JMP^® very easily, and the interactivity is retained (Figure 6). Dashboards, in the same way as other visualisations, can be converted into HTML so they can be explored without the need to have JMP^®. Dashboards allow scientists to present key findings and can support stakeholders with decision making.

Fig. 6.

Once the model is built, the effect of the inputs to the outputs can be visualised using the prediction profiler (Figure 5) which is one of the most powerful tools available in JMP^®. As already mentioned, this allows the scientist to explore the effect of the factors and better understand the chemical problem. It is also a great tool to communicate the process and the effect of the factors. JMP^® allows these visualisations to be saved in an interactive format which can be shared across different functions. An example of utilising these tools to generate value has been demonstrated within Johnson Matthey. When the commercial team received enquiries regarding the use of a product under certain conditions, they had to contact the development team to access the information. The research team has now built a model as a result of a response surface DoE. The model has been shared with the commercial team using the interactive prediction profiler. With this, the commercial team can predict the performance depending on the conditions suggested by the customers. This tool has provided the commercial team with more autonomy and a quicker response to the customer and has saved time for the development team. Statistical tools can not only help us to visualise data but also to make objective decisions.

The use of statistics in disciplines such as physics, biology, medicine and finance is common (2, 3). However, in our experience, its use in chemistry has been sparse despite it being a useful tool, and some would say, indispensable.

The aim of experimentation is typically exploratory, to gain understanding, or to optimise a process. Although the objective might be different, a tool that helps to differentiate between the experimental variability and the effect of a particular input is needed. This is where statistics can help to make more informed decisions. Statistical tests are carried out to understand if results are statistically significant or not. When talking about statistically significant results, we refer to those results obtained by testing or experimentation that are not likely to occur randomly or by chance, instead they are due to a specific cause. Often p-values are used to describe this. Although the inappropriate use of p-values in some cases has brought controversy (4–7), they can be very useful. It is important to remember that the conclusions drawn from statistical tests should be interpreted within the context of the study (sample size, reliability and validity of the instruments used to measure the outputs).

An example of this within Johnson Matthey has been a comparison study between several analysers (Figure 7). The statistical tool facilitated the visualisation and helped to establish the significance of the differences found between the measurements obtained in the analysers when dealing with the same samples. These types of studies are crucial to ensure the reproducibility of results.

As seen so far, the toolbox is quite extensive and sometimes that can be slightly overwhelming. For example, when generating a DoE, it is possible to be intimidated by the choice of design types available. However, JMP^® has features to help when evaluating and comparing designs.

Fig. 7.

The choice of design depends upon the aims of the project (screening or optimisation) and the resolution required (main effects, higher order terms). Classical DoEs (full factorial and fractional factorial designs) are no longer used as often as increasingly popular modern designs (definitive screening designs and bespoke custom designs) (8, 9). The design choice must then be carefully balanced against the resources available (timeframe, cost of running experiments) to decide upon the experimental matrix to be used. More experiments will provide more information about the system, but often this is not possible because of practical or financial constraints. It therefore becomes extremely important to compare multiple designs and understand the relative advantages and disadvantages of each.

This is made possible with the ‘evaluate design’ and ‘compare designs’ tools in JMP^®. Potential designs can be opened side-by-side and comparisons made. Power analysis helps to estimate the ability of the design to detect effects of importance by reporting the probability of detecting effects of a given size. Higher powers for model terms result in a greater chance of detecting their effect. Prediction variance profiling displays the uncertainty across the experimental space and can be altered depending on the focus of the design. For example, an optimisation design would try to minimise prediction variance at the centre of the experimental space. Colour maps of correlations show the absolute value of the correlation between any two effects that appear in the prediction model, represented visually with a sequential colour scheme (Figure 8). This helps to identify where factors and higher order terms in the models may be partially or fully confounded, and where one design might have the advantage over another.

Fig. 8.

The eventual design choice will be unique to the scenario, but evaluation and comparison of multiple designs allows the requirements of the project to be considered against the real-world implications. Running more experiments will provide additional understanding of the system but resource may only be available for a predefined number of experiments. These tools allow the best choice to be made to carry out experimentation in the most efficient manner to maximise the information that can be gained while also identifying the limitations of the design. The efficiency of statistical design has already been mentioned several times, and this characteristic is due to the systematic and structured nature of this approach.

The traditional approach to experimentation, which is still taught in most universities, consists of changing one input while keeping the others constant. This provides the certainty, or so it is believed, that the variance observed in the output is due to this change. However, this approach has many pitfalls: there is no way of studying the interaction between two inputs, experimental error is not accounted for and the experimental space is not fully covered. DoE corrects all these pitfalls: it allows the study of interactions between inputs, the experimental error is accounted for and all the experimental space is fully covered. All this provides more control than traditional experimentation.

The process of carrying out statistically designed experimentation follows a structured approach. Initially, the experimental space is decided by the scientist based on experience or prior knowledge. If working in a new area, a pilot trial can be used to help the scientist. Once the first set of experiments has been completed and analysed, further experiments can be planned based on the results obtained, the aim of the experimentation and the number of experiments that can be performed. Also, experiments to validate the model should be carried out. The scientist has control over the experimental plan and the statistical tools are only there to facilitate the work. All this is made very easy by JMP^® as it provides different platforms to generate the different designs and augmentations. As already commented, tools to evaluate the designs can also be found in these platforms so the scientist can take an informed decision when selecting the design.

As emphasised extensively in this article, there are multiple benefits from utilising statistical tools for product development and process optimisation. However, their implementation has not been so widely applied, especially in the chemical industry. It is worth highlighting some of the challenges and how to overcome them.

Commonly, data logging from scientific equipment and data analysis from experiments is done within Microsoft Excel (Microsoft Corporation, USA). Scientists are familiar with this program, having probably used it daily during the entirety of their career. There is a reluctance to move away from something to which we are so accustomed, in some cases to the point where we can no longer see the limitations. Microsoft Excel is an excellent spreadsheet program with simple user interface, ensuring it is used universally. However, it was never designed with the intention of handling and interpreting large volumes of data. A recent example of its misuse resulted in Public Health England failing to report nearly 16,000 coronavirus cases in 2020 (10). Add-ins are available to perform simple statistical functions, but specialist software like JMP^® is required to thoroughly interrogate data and deliver greater understanding.

As well as the tools available within JMP^® to provide greater insight, it has been purposely designed to manipulate and visualise large datasets. This is exemplified by features such as ‘graph builder’ and ‘dynamic linking’, as previously shown in Figure 4. The click-and-drag interface when building graphs in JMP^® is a much simpler workflow to visualise data than creating graphs within Excel. There is also a JMP^® add-in available for use in Excel which allows the user to transfer data between the two programmes in a single step and quickly access some of the common analysis platforms of JMP^®. From experience within Johnson Matthey, we have found that the key to persuading people away from Excel and into specialist software is to show a direct comparison of a typical workflow with real-life data used in that part of the company. The improved visualisation and data analysis are immediately obvious, as are the time savings, which liberates more time for scientists to develop new technologies and products in the laboratory rather than handling and formatting data. However, there is also a barrier to overcome when learning to use new software.

Johnson Matthey has recognised the benefits of promoting and instilling a culture of advanced data analytics. However, a common barrier to overcome when transitioning to new ways of working is the initial investment of time required to get to grips with new software. For research professionals whose time is a precious commodity, the investment of time needed upfront to learn new techniques and navigate around the software can be deterring. This is especially true as this part of the learning curve does not provide any immediate, tangible output. Furthermore, the wealth and variety of training resources available to new software users can make the learning process seem initially overwhelming. From experience within Johnson Matthey, we have found that setting aside time at regular intervals to progress through a predetermined training plan helps to make the process as simple as possible for new users. The training plan can be developed alongside a more experienced software user and will be bespoke to the requirements of the individual, concentrating on the functionality of the software with which the user will primarily be working. The training plan typically includes different resources, such as individual learning (online webinars, e-learning subscriptions, ‘Statistical Thinking for Industrial Problem Solving Modules’ – a free online statistics course provided by JMP^®) and group learning (Johnson Matthey specific software introduction courses developed and run by experienced users). There is also an active JMP^® user community within Johnson Matthey which was created to support new users, provide an informal environment for sharing knowledge and as an open forum to ask questions about specific problems.

Demonstrations of Johnson Matthey projects to senior management where the software has been used to improve process understanding have been critical to increase awareness of the benefits the software can bring. This has resulted in management encouraging staff to dedicate time towards software training. The impact of coronavirus has also accelerated this process, as developing new software skills is a task that can be carried out while working from home, either during forced periods of self-isolation or by minimising regular operations on site. But obviously it is not all about learning to use new software, it is also about learning statistics.

As already mentioned, the use of statistics is more common in other fields such as biology or medicine than in chemistry. Traditionally statistics is not a featured component in chemistry undergraduate degrees. If some statistical content was taught in the first years, the learning was not normally reinforced with practical activities further on in the course. This can make chemists uncomfortable around statistics.

Within Johnson Matthey we believe that a practical understanding of statistics can be achieved to complement chemistry expertise by our scientists. The use of practical statistics is now totally accessible by using software packages like JMP^®. The learning curve of statistics goes hand-in-hand with learning new software from a practical point of view. It allows scientists to practise and learn with their own data, which has been proved to be the best way to learn, always supported by the most experienced users within the company and learning from each other’s cases. This process is not easy because it requires a total change of culture.

While DoE methodology has been applied experimentally for decades, it is only relatively recently that its usage has gathered momentum across many scientific disciplines. This is due to a combination of advances in the algorithms used to tailor designs to the experiments and an increasing industrial need for rapid experimentation and decision making. For example, addressing design space constraints (11), handling mixture-type factors (12), comparing the effectiveness of different designs (13), introducing uncertainty in the factors and optimising using a variability simulator (14). However, for traditionally trained scientists who are used to changing OFAT in accordance with the scientific method, the transition to DoE methodology can be met with trepidation. There can be a concern that the scientist’s skills are not being fully utilised and that the recommended experiments in the matrix will not be enough to understand the system. Overcoming these anxieties is a significant challenge, particularly within established R&D departments. At Johnson Matthey, the way this has been approached is to demonstrate the power of DoE on small projects across a range of technology areas, and actively promote these results to the rest of the company, increasing the visibility and viability of the DoE methodology. This generates additional interest and establishes confidence in the methods so that scientists have more faith in using DoE for larger, more complicated projects. The functionality of software such as JMP^® to create designs, analyse the results and present the conclusions is essential in facilitating this cultural shift.

At Johnson Matthey, the key principle when driving this transition to an advanced experimentation and statistical approach to data is to empower our scientists to do it themselves. The scientists are the technical experts in their respective areas, and by giving them the understanding, tools and training to create and analyse DoE it is believed that this will result in better outcomes for both the current project and future work programmes.

Indeed, recognising the technical expertise in their respective areas when deploying a statistical software like JMP^® is a very important step to overcome a very important concern; the fear of being substituted by a computer or a machine (15).

The media can be overwhelming in this respect: listening and reading continuously about artificial intelligence, robotics, automation and machine learning. However, technical expertise will always be necessary, and the human being has proved to be indispensable in many fields. Statistical techniques like DoE are not designed to substitute the chemical expertise of a human scientist but to work in conjunction with them to get the most out of experimentation, and to make them more efficient. Statistical tools are exactly that: tools to be used, not to substitute scientists.

Indeed, the first step in a statistical design is the planning. For this step, chemical expertise plays a crucial role. The DoE is not going to tell the scientist which factors or responses should be studied. It is the scientist who should feed all this valuable information into the design.

In the same way, as a result of a DoE it could be found that a variable has or does not have an effect on the output, but it will not say why. It is up to the scientist to interpret the result, try to understand why and continue designing more experiments to test and prove that hypothesis.

When teaching these techniques within Johnson Matthey we are very careful to emphasise these tools are to help scientists, not to replace them. It is crucial to motivate scientists to believe in the process to overcome other major challenges, such as the timings.

Another challenge that scientists experience when using DoE is the lack of immediate visibility of the factors’ effects. In traditional experimentation, the scientists can see the effect that changing an input has on the output once the experiment has finished and then, based on this result, decide the next experiment. However, there is no visibility of progress while carrying out experiments from a DoE as analysis of the results only makes sense once all the experiments of the design have been carried out. This requires some patience and trust from the scientists to see it through. Therefore, the first time that someone uses such tools they will struggle but once they see the results, they understand that the wait was worthwhile. For this reason, it is recommended to start with smaller sets of experiments instead of embarking on a large, complex DoE. Also, to start with a relatively simple DoE such as a full factorial design with only a few factors, to overcome another important challenge, the fear of the ‘black box’.

Another big challenge that pushes scientists away from using DoE is that it is seen as a ‘black box’. A lack of understanding of the technique together with a limited understanding of statistics creates uncertainty and the scientist can feel a loss of control. It is an understandable response and can only be helped by providing the information needed to understand the technique and its benefits.

Work is being done at Johnson Matthey to make sure that scientists are provided with the tools and support necessary, so they can understand the techniques and use them with confidence. Different approaches are taken for this: one-to-one training, in-house and external group training. Also, the use of software such as JMP^® has been critical to empower the scientists at Johnson Matthey to use such tools. The program is easy to use and there are lots of free learning materials available from JMP^®. For new users who do not feel very adventurous, starting with a simple and more intuitive design such as a small full factorial is recommended. Despite starting with something simple, the results will sometimes be unsatisfactory for the experimenters, and might reveal some difficult truths.

The use of statistics and DoE during experimentation might uncover some weaknesses in the way the experiments are carried out. Sometimes, inconclusive results will be obtained from DoE due to irreproducibility issues on the experimentation. It might be tempting to point at the DoE as the problem; however, DoE has only helped to uncover an issue that already existed even when performing traditional experimentation. Instead of seeing this as an issue it should be thought of as an opportunity to improve the way experimentation is carried out and to reduce the experimental error.

The variability observed could be due to many reasons, such as uncontrolled factors that affect the response or its measurement. These could be included in a subsequent DoE to be studied further and help to provide better understanding of the system (Figure 9). To be included in the study, the scientist needs to be able to measure and control the different inputs. Understanding the origin of the variability of an experiment can be used to improve the process. For example, if a more precise measurement of the output can be obtained, the scientist would be able to observe smaller effects of inputs which, combined, could drive larger improvements in the output.

Fig 9.

DoE and statistical tools allow experimenters to obtain reliable data in order to extract objective conclusions and take decisions, even if those conclusions are that the experimentation needs to be redesigned or the measurement system improved.

Pelleting processes are used in the manufacture of a range of consumer and industrial products, including tableted pharmaceuticals, health products, consumer goods, food products and catalysts. In the manufacture of these pelleted products, it is important to produce pellets that are consistent in size, shape, composition, density and strength to ensure that they are fit for purpose. Usually, several of the aforementioned variables will be listed in the product specification along with appropriate bounds that need to be met. Poorly controlled pelleting processes can result in production of out-of-specification material, which is associated with negative economic and environmental impacts, and with potential safety implications in the case of pharmaceuticals for example. Conversely, operating pelleting processes with tight control over the CQAs can allow manufacturers to operate closer to the specification limits and to improve the product, the process economics and sustainability.

To establish tight control over a manufacturing process, it is important to control all the variables in the manufacturing process that have the potential to impact the product CQAs. Before that can be achieved, it is necessary to understand the relationships between the manufacturing variables and the product CQAs and to identify those that are most influential, as well as those that can be manipulated to control the product CQAs (1). Henceforth, knowledge of the interactions between process variables and the response of the process is critical. Typically, models are deployed to help us gain understanding of process behaviour. In the literature, there are three main approaches that have been applied to modelling pelleting process behaviour, these are mechanistic, data-driven and hybrid modelling approaches.

Traditionally, mechanistic models such as the Drucker-Prager Cap (DPC) model and finite element models have been applied to better understand the behaviour of materials undergoing compaction and their resultant physical properties. Wu et al. (2) used finite element methods (FEM) to gain understanding of the behaviour of pharmaceutical powders during compaction. The DPC model was used as the yield surface of the medium, representing failure and yield behaviours. Experiments were carried out using a compaction simulator with an instrumented die to calibrate the DPC model and to investigate the relationship between relative density of the powder bed and the applied pressure during compaction. The DPC model generated realistic powder properties that were fed into finite element analysis (FEA), which was able to accurately model the relationship between relative powder bed density and the compaction force. FEA also allowed close examination of the evolution of stress distribution during relaxation, which revealed narrow bands of localised intensive shear stresses where potential failure mechanisms can initiate. Several other works have utilised purely mechanistic approaches to model pelleting behaviour (3–7).

The drawback to mechanistic approaches is that they take significant human resource to develop and they are not easily adapted to new processes or scenarios. In contrast, purely data-driven approaches are very fast to develop and deploy and are easily adapted to new contexts. Several works have focused on the application of data-driven models to pelleting processes (8–15). Haware et al. (9) applied multivariate analysis to quantify the relationships between material properties of α-lactose monohydrate grades, PPs and the tablet tensile strength. The materials were tableted on a compaction simulator and the collected data were analysed with PCA and PLS regression. PCA provided insights into relationships between different powder and compression properties of the studied materials. PLS was successfully used to predict tablet tensile strength from the compression parameters, punch velocity and the lubricant fraction.

Li et al. (10) used multivariate analysis to evaluate the fundamental and functional properties of natural plant product (NPP) powders and their suitability for direct compaction. NPP powders were prepared by three different methods and data were produced in a single-punch compaction simulator. Results from a one-way analysis of variance, cluster analysis and PCA showed that the physical properties of the NPP powders were mainly determined by their particle structure, which derived from the preparation method. Stepwise regression analysis indicated that the compaction properties of the NPP powders could be improved by controlling physical properties, such as density, particle size, morphology and texture. Overall, the work provided guidance on the development of NPP powders for compaction.

Matji et al. (16) conducted a multivariate analysis on data from the production of ibuprofen tablets. In their study, regression methods were used to predict the CQAs of the tablets, such as disintegration time, dissolution, hardness, porosity and tensile strength, from the pressure applied in roller compaction (dry granulation) and tabletting. Tabletting compaction pressure was found to be positively correlated to disintegration time, tensile strength and hardness, and negatively correlated to the porosity and percentage of drug dissolved. Roller compaction pressure during dry granulation was observed to have those same correlations with the CQAs inversed.

More recent works have also focused on the development of hybrid models for pelleting processes, which combine mechanistic models with data-driven models to leverage benefits from both approaches (17, 18). For example, Benvenuti et al. (17) trained an artificial neural network (ANN) to model the relationship between macroscopic experimental results and microscopic parameters of discrete element method (DEM) simulations. The work showed that the ANNs could be used to generically identify DEM material parameters for any given non-cohesive granular material. Hybrid modelling approaches can offer great benefits where existing mechanistic models are available because they leverage the accuracy and interpretability of the mechanistic model, while offering increased flexibility.

In this work, a data-driven approach is used to model pelleting performance in order to gain a deeper understanding of the process behaviour and to understand the potential of such models for use in process monitoring and control. The work expands upon previous data-driven modelling of pelleting processes by exploring the impact of two different feeder mechanisms: a force feeder and a vibration feeder, operated at different speeds. Furthermore, the product studied is an inorganic catalyst material. Analysis of such materials in pelleting processes has not been widely reported in the literature. The pelleted catalyst product studied is manufactured by Johnson Matthey.

2.1 Equipment

2.2 Experimental Work

2.3 Principal Component Analysis

2.4 Partial Least Squares Regression

In order to compare the effect of the powder feeder mechanism and speed on pellet density and hardness, the distributions of pellet density and hardness were plotted for each experiment and pairwise t-tests were used to check for statistically significant differences in average pellet density and hardness. Figures 2(a) and 2(b) show the distributions of pellet density and hardness, respectively.

Figure 2 shows that the feeder configuration greatly influenced the level of variability in pellet density and hardness. In particular, experiments ‘Left40’ and ‘Right70’ produced a large amount of variability, while the vibration feeder resulted in much less variability at all speeds tested. The pairwise t-tests revealed statistically significant shifts in average pellet density and pellet hardness between the six experiments. For example, with the vibration feeder and the force feeder in ‘left’ orientation, increasing the speed of the feeder resulted in statistically significant increases in pellet density. Importantly, the distributions indicate that the vibration feeder results in more consistent pellet density and hardness than the force feeder, although the force feeder may be used in the ‘left’ orientation at high speed to minimise variability.

Fig. 2.

PCA was used to assess the correlations between the variables in the compaction simulator summary dataset and their contributions to the overall process variability. The PCA model presented here was built on summary data from the experiments using the force feeder, namely ‘Left40’, ‘Left70’ and ‘Right70’. Figure 3 shows the variance explained by each PC in an eight PC model. The first PC explains 49.3% of the variation in the data, while PCs 2 and 3 explain 14.1% and 7.2%, respectively. Collectively, the first three PCs capture 70.6% of the variance in the data, while PCs 4 and beyond explain less than 5% of the variance each. Due to the exploratory nature of this analysis, it is preferable to consider all the PCs that are likely to be informative; however, determination of the number of PCs to include in the model is not critically important. In this work, the first three PCs were analysed because PC 4 and beyond capture very little variance and likely feature a low signal to noise ratio.

Fig. 3.

The scores of the PCA model for PCs 1 to 3 are displayed in Figure 4 and the most important variables – those with loadings of the highest magnitude for each PC – are displayed in Figure 5. PC 1 captures variation in the data that is present on a pellet-to-pellet basis within each of the three experimental runs and the scores overlap, i.e. there is no separation of the scores by experiment on PC 1. Figure 5(a) shows that the 49% of variation captured in PC 1 is attributed to the forces and energies involved in the pre-compaction, main-compaction and ejection events. The model also indicates that the forces and energies listed in Figure 5(a) are all positively correlated with one another.

Fig. 4.

Fig. 5.

In contrast to PC 1, PC 2 captures variation that separates the different experiments. Figure 5(b) shows this is largely attributed to differences in die filling height, ejection force and some derived parameters which are determined from die filling height, such as the compression and relaxation times. The large spread of the red markers in the vertical plane, corresponding to the ‘Right70’ experiment, indicates that this set up resulted in more variability in the die filling height compared to ‘Left40’ and ‘Left70’. The compaction simulator was calibrated to produce pellets of the same weight for each experiment; therefore, it is likely that the different feeder setups result in a different bulk density of the powder when it is initially filled into the die, explaining the differences in filling height.

Figure 4(b) shows the within group variability that is captured by PCs 1 and 3. All of the experiments overlap on PCs 1 and 3, while ‘Right70’ spreads across the largest area, indicating that this experiment has the largest variability on these PCs. Observing the distributions of the response variables, it is clear that ‘Right70’ produces pellets with the largest variation in pellet density and hardness. As shown in Figure 5(c), the main variables represented by PC 3 are the energies involved in the pellet ejection and the elastic energy calculated for the main-compaction event.

While the ejection plastic energy and compression energy correlated with the pre-compaction and main-compaction forces, the ejection energy and the ejection force appeared as key variables in PCs 2 and 3, indicating that that there is variance in the ejection force that is uncorrelated to pre-compaction and main-compaction force. PC 2 shows that ejection force is positively correlated to die filling height. An additional factor that may be influencing the ejection force that is not monitored, so not captured by the dataset, is the amount of lubricant present in the material.

PLS regression models were developed to predict pellet density and pellet hardness, using the methodology outlined in Section 2. Table II shows the performance metrics obtained from cross-validation and testing of the two models for pellet hardness and density.

The performance metrics shown in Table II are for the models obtained after the variable selection procedure and tuning of the number of latent variables included in the model, as described in Section 2.4. The models for both pellet hardness and density performed well in cross-validation and testing. Pellet hardness however proved to be the more difficult to model and predict accurately, as the performance metrics demonstrate. The pellet density PLS model explained approximately 86% of the variance in both 10-fold cross-validation and independent testing. The MAE for the pellet density model was 0.17 and 0.18 in 10-fold cross-validation and independent testing, respectively. The cross-validation metrics indicate that the pellet density PLS model should have excellent predictive performance and the independent test on the held-out data supports this. The fit of the density PLS model to the training data and the testing data is shown in Figure 6.

Fig. 6.

Figure 6 shows that the density PLS model fits the data well in both training and testing, with the exception of a few outliers, which are likely to result from a mismatch between the X and Y data that occurred during the experimental process.

The pellet hardness PLS model explained approximately 67% and 63% of the variance in pellet hardness in 10-fold cross-validation and testing, respectively. The MAE for this model was 0.44 and 0.53 in cross-validation and testing, respectively. While this performance is not as good as the pellet density model, it indicates that the model has good predictive capability. Observation of the measured versus fitted values and the residuals in Figure 7 reveals that the model is good at fitting and predicting the low-hardness pellets but is far less accurate for the high-hardness pellets. The residuals in Figure 7(b) and 7(d) ‘fan-out’ and become much larger from –1 and above on the x-axis (scaled pellet hardness). The increasing residuals with increasing pellet hardness could be related to missing factors that are not captured in the compaction simulator data. It could however equally be the result of changes in the sensitivity of the measurement device at different hardness levels. Unfortunately, it is difficult to gain a good understanding of the reliability of the hardness measurement due to the test being destructive. Given that the model offers accurate prediction of low hardness pellets, the model could still be valuable in a process monitoring or supervisory process control applications, where identification of low hardness pellets could allow operators to intervene early and adjust PPs accordingly.

Fig. 7.

5.1 Interpretation of the Model Coefficients

The workflow for this study began with exploratory data analysis to observe and compare the distributions of pellet density and hardness and to understand the variance and correlation in the compaction simulator data. The distribution plots clearly showed that feeder configuration impacted both the average pellet density and hardness and the level of variability in those properties. Pairwise t-tests revealed that the shifts in mean density and hardness were significant in many cases. To minimise overall variability, the vibration feeder mechanism should be favoured over the force feeder mechanism, however, the force feeder mechanism can be optimised for consistency by running in the ‘left’ orientation at higher speeds.

PCA showed that the most important component of variance in the dataset was attributed to the forces and energies involved in the pre-compaction, main-compaction and ejection events. The main differences between the force feeder experiments that were highlighted were due to the die filling height and associated parameters, and some observable differences in overall variability. The high level of variability in pellet density and hardness for experiments ‘Right70’ and ‘Left40’ was also reflected in the energies and forces recorded in the compaction simulator summary data through the PC scores.

The PLS models that were developed for pellet density and pellet hardness performed well in cross-validation and testing. The key predictor variable in the models for pellet density and hardness was the lower punch force, which correlated positively with both. The density PLS model explained around 86% of the variance in the response, while the hardness PLS model explained around 65%. Both models offer predictive capability, however, the hardness model underperforms at predicting the medium to high hardness pellets. Unfortunately, it is difficult to gain a good understanding of the reliability of the hardness measurement due to the test being destructive. The questions raised in this study highlight the need to further validate the pellet hardness measurement in future work. For now, the learning from this work indicates that the pellet density can be used more reliably as an indicator of product quality than pellet hardness. Pellet density should therefore be the preferred basis for process monitoring and control applications.

If a similar model for pellet density can be developed using the data available from production scale pelleting machines, then there would be the potential for such a model to be used for process monitoring and control. This could provide real-time process monitoring that greatly improves upon existing techniques for monitoring pelleting performance, which are based on random sampling and testing of the pellets. The information could be used by plant operators at a supervisory level or in an automated control system to help inform and guide decision making to keep the process on track and producing on-specification material.