Step 3 · Method
Explain what you did and why it suits the question. That's half of Criterion A.
By the end of this step
- A method plan your supervisor has seen, and pilot results.
Primary, secondary, or both
| Primary data (you produce it) | Secondary data (others produced it) | |
|---|---|---|
| Examples | Benchmarks, simulations, training models, timing your own implementations | Published datasets, results in papers, vendor benchmarks |
| You must | Describe it well enough that someone could repeat it | Make your selection criteria explicit and do the analysis yourself |
The IB guide saysEvery EE needs secondary research: a review of the literature on your topic. In CS, prioritise peer-reviewed journals, conference proceedings and scholarly books. Blogs and websites can give background, but shouldn't be the basis of your analysis.
Design an experiment
| Part | Hash-table essay |
|---|---|
| Independent variable (what you change, with levels) | Load factor: 0.1, 0.2 … 0.9 (9 levels) |
| Dependent variables (decide before you start) | Mean lookup time (µs); mean probes per lookup |
| Techniques compared | Separate chaining vs linear probing, both written by me in Python |
| Controlled variables | Same 100,000 keys (fixed random seed), same hash function, same laptop on mains power, no other apps, same Python version |
| Repeats | 20 runs per level after 3 warm-up runs; report mean and standard deviation |
| Why this method | Probes measure the algorithm itself; time shows the real-world cost including memory effects |
The hash-table essay is a made-up example used throughout this guide. Write about your own question, data and decisions.
Independent, dependent or controlled? Self-check drill
Classify each variable in the hash-table experiment.
Keep everything in the MIDST constant Machine · Input data · Distractions · Software versions · Timer
Same machine on mains power; the same input data (fixed seed); no other apps running; the same language, library and interpreter versions; the same high-resolution timer.
Same machine on mains power; the same input data (fixed seed); no other apps running; the same language, library and interpreter versions; the same high-resolution timer.
Why repeats matter
Can you tell two algorithms apart? Simulator
A is truly 0.8 ms faster than B, but every measurement is noisy. Change the noise and the number of repeats, then press Measure several times. When can you honestly claim A is faster?
Real life: benchmarks that lied In 2013 the technology site AnandTech found that the Samsung Galaxy S4 raised its graphics clock speed only when it detected popular benchmark apps, so its scores didn't reflect normal use. In 2022 Geekbench delisted Samsung's Galaxy S22 phones for throttling ordinary apps but not benchmarks. Lesson: make sure your test measures what really happens, and say how you know.
Experiment design labPlan your variables and controls, then time real sorting code in your browser.In what order?
Running an experiment Order drill
Put the steps of a fair experiment in order.
Common traps in CS experiments
Timing code
- Use a high-resolution timer (e.g.
time.perf_counter()in Python), not the clock. - Time only the algorithm: not file reading, printing or data generation.
- Do warm-up runs, then many repeats; report the mean and the spread.
- Close other programs, use mains power, and run every condition on the same machine.
- Count operations (comparisons, probes, swaps) as well as time: counts don't depend on your hardware.
Machine learning
- Same pre-processing and the same train/test split for every model; fix random seeds.
- Never let test data leak into training or tuning.
- Choose metrics that fit the problem (accuracy alone misleads on unbalanced data: add precision, recall, F1).
- Use a documented public dataset and cite it.
Security and networks
- Compare at equivalent security levels (e.g. 2048-bit RSA vs 256-bit ECC).
- Use one well-known library for both techniques so the comparison is fair.
- Only test systems you own, in a test environment. Never probe real networks or services.
My code ran twice as fast the second time.
Turns out I'd plugged my laptop in. That's a controlled variable now.
Your code
- Code exists to answer the RQ. The body of the essay discusses results, not how you wrote the program.
- Put code in an appendix. Examiners don't have to read it, so nothing essential can be only there.
- Libraries are fine: name and cite them, with versions.
- If an AI tool helped generate a dataset or test harness, acknowledge it.