Solving Complex Problems

"A famous explorer once said, that the extraordinary is in what we do, not who we are." (Tomb Raider)

Solving complex problems is something that has been addressed in many other places, most likely in better and more thorough ways. It will be helpful to look at existing literature/tutorials/videos on problem-solving. This page is just meant to give some general advice for solving complex problems, why complex solutions are needed for solving complex problems, how problem-solving evolves from a simple to a complex solution and some general tips.

The example of Anti-Spam

To understand complex problem solving, we can look at the example of anti-spam. Assume we're in the era when spam just started. At the time spam was simple and a set of a few rules was enough to deal with most of it. As time went on, spamming methods changed and evolved and anti-spam had to change and evolve in response. New spamming techniques were observed over time and anti-spam was adjusted accordingly. Compared to original anti-spam rules, today's rules are complex. It is the only way to deal with spam of different types that use new spamming techniques. No matter what, sometimes valid email gets marked as spam and that doesn’t mean we should get rid of anti-spam rules. People accept a solution that works most of the time and fails for a small percentage of the time. We can take the example of a tree. There are many branches and paths. If the anti-spam solution in the early era of internet was compared to the today's solution, we can compare a small plant to a large tree which has grown over time.

In today’s era, we cannot use simple black/white rules to deal with spam. This would result in failure to catch a lot of spam and a lot of false flags. A complex system is necessary. Likewise we cannot use simple methods to solve complex problems in other areas of life although our solutions will start simple. As we work on the solution, it will get more complex with time.

Effective approaches for solving complex problems

Similarly, complex problems in the real world can only be solved using a multi-faceted complex system that starts out simple and evolves continuously over time as new problems are observed.

Complex problems have complex solutions and almost always there will be some cases which cannot be treated as intended and that is ok and it doesn’t invalidate the solution.

We can also understand that the application of today's solution to the early era of spam wouldn't be possible because the solution hadnt been invented yet. We just have to be patient with the problem-solving process and know that the solution is always evolving and better better. There may be times where it makes things work but thats ok as long as the general direction is correct.

We cannot reject a solution because new problems are observed or because there are some situations where the solution doesn’t work. We have to be willing to evaluate multiple solutions at the same time. We have to expect that new problems will always come up during problem solving. One wrong and short-sighted approach is to reject a solution because it does not immediately solve the main problem. The correct approach to complex problem solving is to be persistent and explore all possible options and paths and only give up on a certain path when we know for sure we have exhausted all possible options.

A solution should not be rejected because it does not apply in another separate situation.

Solutions should not exclude satisfying sizable segments of the population. They should be applicable to a wide range of people, not just to a small segment of the population. In other words, multiple solutions should be implemented to cater to different target audiences.

Complex problems take a long time to solve and they are often solved in stages. Sometimes a problem takes 10’s of years to solve and may involve multiple generations. For example if a solution has 20 different stages we cannot implement stage 7 before implementing the previous stages. Often times people resist implementing big changes because they feel threatened in some way or they disagree with the solution for whatever reason. We have to be careful. We must work on the first stages of a problem and we have to be cautious about trying to implement certain solutions too early. With problem-solving comes the necessity to understand the environment and the people involved, the variety of opinions that exist and how to best work as a team in the forward direction at the best possible speed and not risk alienating team members and segments of the population and therefore impeding progress.

Some Aspects of Problem-solving

• Presenting data, research or other relevant information from other parts of the world where more effective solutions have already been discovered/ implemented
• Standardization

Maintaining a balance of Complexity

If a system is too complex beyond what is necessary, participants get tired of it and will not want to use it. Therefore its important to keep a balance while designing a system to fulfil its requirements in the simplest way possible. The design of every element has to be done carefully keeping this in mind and should feel user-friendly and intuitive. Every effort should be made to find out if there are any elements that can be merged or simplified.

Finding Existing Solutions

Chances are that a partial or full solution already exists. We need to have a way where we can motivate people to research and search for solutions implemented in other parts of the globe.

Problem Tree

Any complex problem can have one or more solutions and usually there will be one solution that works the best. The main problem can be broken down into smaller sub-problems which have to be solved. We need an interactive, graphic interface to help visualize problems and understand them better. People can define a main problem and suggest solutions try the different paths to the solution and figure out which path is the best and leads to the best solution.

The diagram below shows a problem tree. A problem (P1, red in the center) has 3 viable solutions (S1,2,3 colored in green). Intermediate solutions are denoted by orange nodes. Grey nodes are dead ends, meaning no further progress was possible from those nodes. There may be one best solution. It is up to people to decide what these paths are and which solutions are the best. As an example, S2 is assumed to be the best solution for this problem.

In most situations, problem P1 is only one node in the problem tree. There will be multiple problems connected to each other. Different solutions to one problem affect how other problems can be solved.

Also all the nodes are that created along the way to the solution help reduce the failure rate. For example we're trying to solve the problem of corruption in medical insurance. Every solution we think of will have a loophole which is another sub-problem. Once that sub-problem is solved we've reduced the failure rate. We can keep going until we feel satisfied with the failure rate. It doesn't have to be 100%. It could be 80%, or 98% etc.

General Tips

Here are some general tips for problem solving:

• The final solution doesnt have to be perfect. On that note, no solution is final because we can always improve it later.
• Problems can be broken down into smaller pieces.
• Take your time to define the problem thoroughly and clearly before trying to find solutions. Define goals (desired outcomes).
• Accept that others will always have views that are different from yours. Don’t let that affect cooperation and teamwork.
• Focus on problems at hand and minimize interpersonal conflicts
• Understand differences of opinion and think about how to work as a team while keeping your eye on the goal. In short, we have to be persistent, flexible and open-minded for finding solutions.
• Be persistent and believe that solutions can always be found or things can always be made better.
• Realize that complex problems have complex solutions
• Detach your personal biases and opinions from the situation. Think with an objective rational mind.
• People often become defensive and emotional when told by others that something they created has problems. Instead of thinking with their heads, they think with their egos. They'll make any excuse to avoid additional work and responsibility for the way they designed their creation. To reduce the chances of this happening, make sure you report the problem thoroughly so the creator of the problem cannot slip out or dodge what is coming. Describe the symptoms, what doesn't work and how it should work. Be sure to keep the approach objective and keep the focus on the problem, not the creator or the person who can fix it.
• Keep your eyes on the goal. Don't let anything distract you or affect your energy.
• Complex problems require long-term and short-term goals.
• Take care of yourself, take breaks and make sure your mind is refreshed when you need it.
• There will be some cases where the current solution doesn’t work. Treat that as a separate problem. Break down bigger problems into smaller pieces. A problem/solution ‘tree’ can be created where sub-problems are treated separately successfully. After we’ve tried our best, we’ll still find some cases where the solution doesn’t work. That is ok and is not a reason to invalidate the full solution tree.
• Different segments of the population may need different solutions. We don’t have to use one solution scenario for everyone.
• Are there are any people or groups of people who are opposed to the solution? Design the solution in such a way that its least likely to be opposed. Often this means designing a better solution which benefits everyone uniquely in different ways.
• If you're passionate and strongly determined to solve the problem, you will have no issue preparing for a long fight if necessary. You realize that the result will be worth it.

Links

• https://solve.mit.edu/ - From MIT (funding available)
• Future planning is critical when solving problems: How to be a good ancestor - "A new political movement makes policymakers imagine themselves as members of future generations." -- "It turns out that when they’re acting as people from the future, they advocate far more transformative policies, from health care provision to climate change action."
• http://sppd.thegovlab.org/ - Solving Public Problems With Data (A series of online lectures by leading experts from the Governance Lab)
  • Also see their other link: Innovation Skills Accelerator - "free, online program to train public entrepreneurs to use innovative methods, including design thinking, evidence-based decision-making and collective intelligence, to solve public problems."
  • Another link from theGovLab: https://www.SolvingPublicProblems.org/ (also only educational materials on the topic, not actual projects or tools)