SRDR 5.1. General Principle for Structuring Questions – SHP Methodology and Statistics Support Team

5.1.1. How to Structure Questions

After you have decided which information you need to extract, then you need to think about the best way to organize or format the question to capture the information in the optimal format for analysis. First, a general principle:

In general, when possible, it is best to use structured rather than free text questions.

There are several options for how to structure questions in SRDR+. Let’s talk about each of them and what they are most useful for.

Text Field: This option is useful for free text entry when the type of information in the article is highly variable and unstructured. For instance, you may want to pull in narrative from the article on how blinding was carried out (since there are many ways to blind a study). So, for instance, a simple text box could be used to capture information presented in the following example.

However, let’s say that we planned a sub-analysis to determine whether the type of blinding had any effect on the outcomes. In this case, we may not want the information in text format, but want to create a more structured question that will allow us to add type of blinding into a subgroup meta-analysis. In this case, we could use the following (which used the Checkbox structure:

Numeric Field: This is a free-text entry field but can only be used to enter numerical values (no alphanumeric entry allowed).

Checkbox (select multiple): This option is good when multiple characteristics may be present in the study (as in the different types of blinding in the above example.

Dropdown (select one): This option is similar to the Radio Button option (below) and is useful for when there is only one alternative possible and when the list of options is fairly restricted (e.g., Yes/No). This is what it would look like on the data entry screen.

Radio button (select one): Like the Dropdown, this option is useful when there is only one alternative possible, but unlike the Dropdown is more useful when there are several possible options (which would be awkward with a Dropdown). A radio button option would look like this.

Select One (with write-in option): If you want to offer the extractors the use of a Dropdown with the flexibility of also typing in another option, you can use this question format. However, you do not want to use this format if the write-in information is anything more than a couple of words. If you want the space to allow extractors to enter more detail when a different option is necessary, use the radio button question with an “other” option and then place a free-text question in a second column (we’ll show you how to set up more than one column or row in a question later).

Select Multiple (with write-in option): This question format allows the data extractor to select more than one option (like the Checkbox type question) as well as write in an option not listed. As with the Select One (with write-in option), do not use this format if you want to allow exactors to enter text more detailed than a word or two.

Question Structure FAQs

How do I know which options to allow for Radio Button or Checkbox options?

Being able to create good Radio Button or Checkbox options assumes that you already have some knowledge of the most common or most appropriate options before you begin to structure the question. This means that in the full-text review stage of article selection, you should be looking for common patterns in options across the articles. So, for instance, if you are answering a diagnostic accuracy question and there are three (or so) common types of test device manufacturers, take note of those and create a structured question to allows the extractor to choose among them. Allowing for free-text entry in this situation will mean that you will have to go back at the analysis phase and reclassify the device manufacturer entries into discrete categories. Save time at the end by structuring the questions up front when at all possible.

When might it be best to use multiple rows or multiple columns in a question?

SRDR+ allows for great flexibility in structuring how questions appear. Sometimes, you may not want a whole series of separate questions, but for multiple sub-questions to be “fit” within a single question. For instance, look at this example of a single question with multiple sub-questions on each row.

We can also allow for multiple columns (this is how we created the “Other” free-text entry fields in one of the above examples). In the following question, we have both multiple columns and multiple rows.

So, how do I create questions with multiple rows or columns?

The question builder tool allows you to add additional rows or columns or both.

A word of caution about using multiple columns: there is too much of a good thing. Adding too many columns can make the data entry screen scroll right. This will make it very difficult for the data extractors (especially on small screens).

I just created a question and went to preview it, but I don’t see the new question. What did I do wrong?

On the page where you will add questions to the particular tabs, you will see in the third column of the table the key questions that this question applies to. If you do not select one of your key questions when constructing the question, then you will not see the question on the data entry screen preview. Extractors will not see the question either!

You can fix this problem by selecting at least one of the key questions at the top of the page.

Bottom line: Because data entry questions are linked to Key Questions, if a Key Question is NOT selected for a data entry question, then you will not see that data entry question show up on the data entry screen.

5.1.2. Piloting Your Questions for Extractions

Finally, an important thing to keep in mind is that you will almost certainly NOT create the perfect data extraction template on the first try. Sometimes you’ll realize that there are questions (or options within questions) you should have included but didn’t. Sometimes you’ll find that the data extractors find the structure of a question confusing.

So, all data extraction templates should be pilot tested by different team members. You can do this by assigning a couple of articles for extraction and then obtaining feedback from the team members. If you are extracting alone, plan to extract data from two to three research studies (making alterations in the extraction template), before you have a final extraction form.