Hints and FAQs
Creating a Design
Entering Missing data
Leave it blank; only use a “0” if a zero is representative of the true outcome.
Only numeric data is allowed in the response column.
Changes to the design
Many things on the design layout screen can be changed by right-clicking on the factor column headers and selecting Edit info…
Right-click on the row header to delete a run or change its status to ignore, highlight or verification.
It is often easiest to rebuild the design if you need to make changes to how the design was created. Click on File, New Design and click YES on the “Use previous design info?” dialog. Doing this preserves all the factor names and levels and the response names. Make the necessary changes and re-build without re-typing everything.
If only factor names and numeric factor limits are being adjusted, use the Column Info Sheet view from the Design Layout screen.
Can’t run one combination of factor levels
If there is only one combination that is difficult or impossible to perform, you might just leave that run out of the experiment and treat it as missing data. Another option is to build a candidate set containing only the possible combinations and building an optimal design. Combine factors if a particular combination of two factors cannot be part of the design. You lose the interactions but can still test for differences (effects) between the combinations. Optimal designs can also have constraints applied to them if the excluded combinations only involve numeric factors.
Messed-up coding levels
This can happen if the factor settings in the design are replaced (over-typed) with substantially different settings as used during the experiment.
Follow these steps:
Right-click on a messed-up factor column header
Choose the Recode option from the list.
Click Yes on the dialog.
Repeat for the other messed-up factors.
Unfortunately, this trick won’t work with mixtures. Copy the current data set, rebuild the design to match the true component ranges and paste the data set into the “new” design.
Planning a two-factor experiment
When an experiment only includes one or two factors, there must be replicates. When the results of all the two-factor combinations are only gathered one time each there is not enough information (degrees of freedom) to test all of the effects. Effects can be estimated (coefficients, but no p-values), but they are based upon the smallest sample size possible.
A single additional replicate of the two-factor design is enough to allow the estimate of a p-value. More design replicates will serve to further increase power of the design.
This advice applies to two-level and multilevel categoric designs that only have two factors.
Too many runs on Multilevel Categoric designs
Multilevel Categoric designs provide a design that has all possible combinations of the factor treatments. When the factor have many levels or there are a large number of factors there are too many runs to allow for an efficient experiment.
Optimal designs are the correct method for reducing the combination to only those necessary to fit a reasonable model. Optimal factorial design by default are designed for a two-factor interaction model. By leaving out the ability to estimate the three-factor and higher order interaction many runs can be saved.
In a design with categoric factors, each treatment must be used at least once. If the optimal design is still too large, then some of the treatments and/or levels will need to be dropped from the design to fit your budget.
How Many Levels Do I Need?
When working with numeric factors, the true response surface is assumed to be a continuous function across the area of interest. Continuous functions can be approximated through a Taylor polynomial. The number of levels required depends on the order of the polynomial you believe will best represent the true response surface’s shape. If you believe the shape of the response surface is a hill, valley, ridge, or saddle then a quadratic model only requires three levels (extreme low, extreme high, and in the middle). If you believe the surface is wavy, then four levels are required for a cubic function. By default, these levels are evenly spaced across the testing range, but are not required to be evenly spaced.
More levels allow the analysis to fit higher-order polynomial models. If a higher-order polynomial is not necessary to model the trend in the data, then more levels does not help fit the correct model. Optimal RSM designs include lack of fit points, which add a few more levels than required to fit the model. This provides a check to make sure the model fits throughout the area of interest.
Designed experiments are used to fit models to data; the models for several responses are combined to optimize the process. Designed experiments are not intended to be used for a brute force approach, where many possible combinations are tested and the winner is picked.
Unless the build is restricted to discrete or categoric factors, optimal designs may have many more levels per factor than are actually necessary, but only a select few combinations of those levels will be in the design.
Minimum Number of Levels to Fit a Polynomial
|
Polynomial |
# Levels |
|---|---|
|
Linear and Interactions |
2 |
|
Quadratic |
3 |
|
Cubic |
4 |
|
Quartic – 4th order |
5 |
|
Quintic – 5th order |
6 |
Putting Data In and Getting Results Out
Importing/Exporting Data
Data can be imported and exported by using the standard copy and paste operations common to PC-based operating systems. Highlight the group of cells and select copy from the Edit menu or right-click in the highlighted area to select “Copy” (Ctrl-C). Go to the program you want to paste to and select paste from the second programs Edit menu (Ctrl-V).
Data sets can be imported from a spreadsheet using copy and paste. A design (usually historical) must be set up in Design-Expert to receive it. Make sure that each factor’s levels, ranges and order match the existing data set. Duplicate or Delete rows in the design layout screen to get the desired number of rows. Select the first cell in each factor and response column (and sometimes Run column) and then paste the existing data set into the design layout.
Designs can be exported from Design-Expert to tab delimited text files. These files can be edited to enter responses and then re-imported by Design-Expert.
The steps for using this feature are…
Create design in Design-Expert.
Save the design.
Export the design as a text file.
Open the text file using a third party text editing software.
Enter the response data using the third party software.
“Save as” a tab delimited text file from the third party software.
Open theo *.dxpx file saved above.
Import the text file with the responses included.
As of version 9, the files created by Design-Expert are XML formatted text files. With XML parsing these files can be directly edited. Several scripting languages and Excel® can correctly parse XML files.
The export/import and XML features are most often used with automated experiment equipment.
Analyzing historical data (data mining)
Data gathered without a designed experiment can be evaluated and analyzed using Design-Expert® software. A “design” must be created in Design-Expert to hold the existing data set.
Historical “designs” are available for response surface and mixture data sets. Enter the minimum and maximum settings for each factor, and the number of rows in the existing data set. Copy and paste the data set into the historical design.
Use the evaluation node to explore the aliasing structure, correlation of the factors and the fraction of design space on the graphs tab. If everything looks okay, continue with the analysis.
Follow this link to our free webinar on the subject of analyzing historical data for some details about the things that can go wrong with historical data analysis.
View alias structure for pre-existing design
Working with existing data sets can be tricky. If it proves difficult to get useful analysis or warning messages about aliased terms appear, click on the Evaluation node on the left side of the screen.
Stat-Ease recommends evaluating the two-factor interaction (2FI) order first as this is where most of the important model terms will come from. Click the Results tab. If it shows no aliasing, then go back to the model tab and increase the order. Repeat until aliased terms appear (if any) to determine the capability of the design.
Fitting more than one model to a response
Only one model may be fit to each response.
If it is necessary to compare two models for the same data,
right-click on the response column header and insert a response;
left-click on the original response column and copy the data; and
left-click on the new response and paste in the data.
With two (or more) columns of the same data, a different model can be fit to the response and the analyses compared at each step.
Journaling
All reports and graphs can be exported to Microsoft Office® programs Word® and PowerPoint®. Select the section of the report or click on the graph that is to be exported, right-click, and select the export method.
Make sure to click on a cell containing text or select the whole report (ctrl + a) to export every section. For graphs just make sure to click on the graph before exporting.
The annotation on the Evaluation and ANOVA is not exported via the journal. Annotation can be copied and pasted separately.
If Word® is already open, the exported object will be inserted at the cursor position. If exporting to PowerPoint® a new slide will be added to the presentation. If the selected program is not open, a new document will be created.
If multiple sections of a report are exported they will be in separate tables and/or on separate pages as necessary to maintain a readable font size.
Making Output Look Nice
Formatting Data
There are global preferences that set the default number of decimal places. These are accessed by going to Edit => Preferences and opening the Math Preferences. Under Math Display, there are options to change the number of significant digits that are displayed.
For the design layout, local format controls are found on the Design Properties tool. This control can be used to over-ride the significant digits and format for each column rather than using the global defaults.
Graph Preferences can be set globally under Edit => Preferences or by right-clicking in the graph area to access the Graphs, Colors, and Font Preferences pages.
Display problems - lines breaking on reports
If the text on a report is breaking up, use the “maximize” icon on the top right corner of the screen.
There may be problems if the resolution of the monitor is set at 640 x 480 or if Windows is set up to use “large” fonts and the monitor resolution is 800 x 600. There are two solutions:
Change to a higher resolution monitor setting.
Change the Windows display to “small” fonts.
Interpreting the Analysis
No significant effects
When nothing is significant, try plotting the factors individually versus the response by using the Graph Columns node on the left side of the screen. You may pick up on some trends that are interesting, even if they are not statistically significant
Green triangles on Half Normal Plot
These triangles represent unused degrees of freedom not used to estimate an effect. The pure error differences between replicates is the most common source of green triangles. The information from the unused degrees of freedom is pooled with the natural process variation to help separate the few true effects from the effects that are just noise.
Watch out for the situation when the green triangles are higher and farther to the right than the largest effect. The estimated error is larger than the effects caused by the factor changes.
Working with Factorial Screening Designs
Screening designs are used when there are so many factors the number of runs required to estimate higher-order terms exceeds the budget.
The goal of the screening experiment is to use as few runs as possible to decide which factors will be studied in more detail during the next phase. The assumption is the factors with strong main effects are the important factors. Saturated resolution III designs are too small for this purpose unless there are in fact no interactions between the factors. Resolution III designs confound the main effects with the two-factor interactions. Fractional factorial designs that are at least resolution IV (yellow, green, or full) are necessary to correctly identify factors with strong independent effects when interactions are likely. Minimum Run Screening and Definitive Screening designs are specialized screening designs that have fewer runs, but less flexibility.
Factors that are known to effect the process outcomes should be held at some reasonable constant and not included as a variable in the experiment. Why spend the runs testing these factors when it is known they have effects and will survive the screening process?
Important factors will change the response a substantial amount. The meaning of substantial is up to the experimenter. Building a design that has at least 80% power to detect substantial effects given the expected variation helps to ensure the screening design has a reasonable chance to find the important factors. However, size of the effects is a more important metric than a significance test on the ANOVA. If no effects are significant, then force the linear terms into the model. Calculate the size of the effect by doubling the linear coefficients shown in the ANOVA. The factors with substantial effects survive to the next round. The perturbation plot is a good visual tool for showing the relative size of the effects.
Screening designs do allow for estimating the interaction effects, but do not usually have enough information to test them for significance. This means the choice of which subset of the interactions encompasses the truth is usually not clearly revealed by the data alone. Informed opinions are a necessary part of analyzing screening designs especially where interactions are involved.
Screening designs are intended to be followed by a characterization design including the previously known to be important factors and the factors that survived screening. Characterization designs concentrate on finding the important interactions and, if center points are included, detecting curvature.
Working with Mixture screening Designs
Mixture screening designs are used when there are so many components the number of runs required to estimate higher-order, blending effects, terms exceeds the budget. One must make the assumption that the linear effects will be enough to decide which components truly matter to the process.
The goal of a mixture screening experiment is to use as few runs as possible to decide which components will be studied in more detail during the latter phases of the experimentation process.
Mixture screening designs require at least 2 vertices per component to estimate the linear effects. Usually replicated center points are added to the base designs to provide a more powerful test and allow the detection of substantial blending effects. Although a mixture screening design can detect blending, there is usually not enough information to determine which components are contributing to the blending effect. The choice of which subset of the components take part in the blending requires the informed opinions of subject matter experts.
The analysis concentrates on estimating the linear gradients and effects of the individual components. These estimate are shown on the ANOVA any time a linear model is fit during the analysis of a mixture design. The Trace plot is a great visual representation of the relative effects coming from the components.
Components that have about the same gradient across all the critical responses can be treated like a family and combined into a single component for future work. This decision must be supported by subject matter expert opinion. If there is no science to justify combining components, then they shouldn’t be combined.
An example of combinable components is blending flours to make cupcakes. Three flours are used cake, bread, and all-purpose – these flours have about the same effect on taste and texture for this example. Because they are all flour, in makes sense that flour blends could be used in place of single flour. A very important part of combining factors is the similarities must carry across all critical responses.
When there are minimize and/or maximize goals for the responses, components that drive the response the wrong way can be set to the minimum possible for future work. If components have little to no beneficial effect on the response (gradient close to 0) these can also be fixed to nominal values for future work. Fixing the setting of a component removes it as a variable from future work.
Combining components into families, removing or minimizing detrimental components and fixing the levels of non-involved components accomplishes the goal of simplifying future work.
Coping with Unusual Outcomes
No lack-of-fit info
The Lack-of-fit test requires replicates, plus more design points than the number of coefficients in the model.
If there are replicates and extra runs, but no lack-of-fit information then most likely the replicates did not vary. It could also mean extra terms were added to the model. These extra terms use up the remaining degrees of freedom.
Predicted R-squared more than 0.2 lower than Adjusted R-squared
Usually this indicates there are too many insignificant terms in the model. As the number of coefficients in the model approaches the number of design points the predictions can become unstable.
Check the residuals versus prediction plot in the diagnostics for outliers.
If no outliers, check further into the diagnostic plots for highly influential observations that may have undue impact on the model.
Sometimes this just happens with no negative impact on the analysis or the final conclusions. It is more of a warning sign than a real problem.
Negative Adjusted R-squared
A negative Adj. R-squared is produced when the MSmodel is less than the MSresidual. This is not a model one would ordinarily choose. A negative Adj. R-squared is a warning that the variation explained by the model (MSmodel) is less than the residual variation (MSresidual). If the model selected is statistically significant (MSmodel is greater than MSresidual) then the Adj. R-squared cannot be negative.
Zero desirability in numerical optimization
If you have responses that completely conflict with each other, i.e., the best settings for one are the worst settings for the other, then it is possible that no region satisfies your criteria. Optimization is a trade-off: The criteria may need to be adjusted so that some acceptable operating region can be found.
Different results than outside calculations
Differences between program results and outside calculations are normally due to a loss of significant digits. These differences can be particularly pronounced when a transformation is being used or when you are using the actual prediction equation. For readability, the numbers are only displayed with a limited number of significant digits. Sixteen significant digits are stored on all numbers and will copy and paste correctly.
When All Else Fails
Try Right-Clicking
Tables, reports and graphs can be directly Exported to Microsoft Word® or PowerPoint®.
Cells on reports have context-sensitive help about what they contain.
Copy the selected contents.
Editable cells can be pasted over or cleared.
The design layout screen has several different right-click functions.
- The column headers (gray squares labeled Run, Factor, Response, etc) give access to sorting, inserting, ignoring, deleting, etc.
- The row headers (gray square to the left of each row) gives access to duplicating, deleting, setting custom row statuses, etc.
- Editable response cells can be toggled between normal and ignored status.
Several adjacent columns, rows, or cells can be selected (click on the first object, shift+click on the last) and then right-click on one of the selected headers to apply the same function to all.
No matter how the above objects are selected, the right-click menu is context sensitive depending on where you right-click.
The Select button on the Design layout spreadsheet.
- Change which columns are currently displayed
- Change the columns displayed by default when a new design is created.
- Sort the design by Row Status.
Model editing screens (where you see “
” and “
” beside the list of factors) right-clicking assigns a term to the model, or error pool respectively. On some models, terms can be forced as well.
On graphs the right-click menu is context sensitive.
- Right-click on the general plot area to change how the graph is displayed.
- Open the Graph Preferences dialog to adjust the X and Y axes tick marks, colors, etc.
- Add a Flag to cube, contour and 3D plots.
- Right-click directly on a flag to change how it is displayed.
- Add a contour to contour and 3D plots.
- Right-click directly on a contour to manually set the value.
- Right-click on a factor on the Factors Tool to assign it to a display axis.
Stat-Ease Direct Support
Support Website
Support inquiries require a valid annual support and maintenance agreement. This agreement is included with annual network licenses.
Please provide the software serial number (formatted as XXXX-XXXX-XXXX-XXXX) which can be found by clicking on the Help Menu and selecting About.
Please also supply the name and address of the organization where the software is being used.
Serial numbers with an active annual service and maintenance (ASM) agreement receive limited support.
If you think you have spotted a bug, or have questions about installing, activating, or licensing the software, contact [email protected].
For questions about how to choose a design for your experiment, interpret the results, or how to use the software contact Statistical support.
If you have a file created by Design-Expert that is related to your question, please attach it to the email.
Q:請教一個問題 , 為何以Design Expert 跑RSM會得到多重的反應值(多解),這是程式設定問題嗎?還是有解決辦法呢?
A:一般程式會給 10組解 讓您選擇.
Q:不知道Design Expert可不可以將方程式輸入,畫三角立體等高線圖?
A:可以 , 說明如下 , 範例請洽本公司.
To force Design-Expert to graph a specific model,
enter it as an equation only model.
Right-click at the top a response column and insert a new response.
Right-click at the top of the new column and select "Equation Only".
Clickon the Model button on the dialog box and enter your equation.
This will NOT fit a model to your data.
But will get you the response surface picture you want except for the design points.
您也可以在反應曲面內 將方程式輸入,畫立體等高線圖.
