### On what measure, do you compare two different sources of suffering? **Why is it important to answer**: to prioritize [[Project ideas for reducing suffering]] that maximize reduction of suffering in the world, we need to know what metric to measure suffering on. Since suffering is a subjective, there's no possibility of an objective measure for it. Until we deeply understand the [[Causal mechanisms for consciousness and suffering]], the only way we can get data for measuring suffering is by asking people about it. #### Objectives for a measure of suffering Following objectives have to be fulfilled for any measure of suffering: - **Allows for ranking/ordering between different types of suffering** (validity => should measure what it's saying to be measuring) - In order to prioritize, we want to be able to ultimately compare which type of suffering is greater than the other. Intuitively we know that having a common cold is a lesser degree of suffering as compared to, say, migraines. But we want this intuition to emerge naturally from the data. - **Calculatable for suffering of all sentient beings** - Beyond humans, there are many [[Sentient beings that are capable of suffering]]. Our measure should inform us about intensity of their suffering as well. Obviously, the key challenge is that we can't ask them about their suffering. - **Should have less measurement variance** (reliability => similar numbers on repeated measure) - The metric for any particular type of suffering shouldn't vary a lot from person to person. This is because in many calculations where we're comparing causes, we probably need one number per type of suffering and if this number varies a lot, taking average or median of it becomes unrepresentative of the distribution. - **Should be cheap to collect** - Since we will need to collect this measure for many types of suffering, it shouldn't be prohibitively expensive. - **Standardizable, that is it should not depend on the data collector* - What data we collect should minimally depend on the specific person who is collecting the data. This is because of two reasons: - Different data collectors should agree on the same conclusions gathered from the data on the same type of suffering - To scale data collection, the art of measuring suffering cannot just be retained by a few individuals. - **Should have reliability #### Proposal for a measure All [[Types of suffering]] ultimately lead to an undesirable state of mind. People do many things to get rid of this undesirable state of mind. In fact, it can be said that the basis of all human activity is to seek ways out of these states of minds, towards positive state of minds. To measure the intensity of undesirability of the state of mind, we can ask or observe what lengths to people go to in order to get rid of the situation they're in. ##### Comparing observing v/s asking **Observe (revealed preferences)**: - Pros: - Closer to truth (of intensity of experience). This is because people's intentions are revealed truly by their actions. For example, many people say they want to lose weight but then they never join the gym or eating processed food. - Works for non-human sentient beings since we can't ask them anything but we can definitely observe them. - Cons: - Actions depend on availability of fixes of suffering. Some people may not act towards fixing their suffering because they know there's no known solution for it (e.g. migraines) - They may not have money to fix their suffering. But that doesn't mean suffering doesn't exist or isn't very intense. **Asking (stated preferences)**: - Pros: - Doesn't depend on availability of fixes of suffering. We can ask people to imagine a perfect solution/fix for the suffering exists. - Doesn't depend on economic situation. Since no money is needed to imagine, we can ask people to imagine they had the money to afford solutions/fixes for their suffering. - Cons: - Doesn't work for non-humans _directly_ as we can't ask them anything. Though we can ask humans to imagine being in the situation that non-humans find themselves in. - If no real stakes are involved, people's personality to exaggerate or underplay can bias the answers. As per [this talk](https://www.youtube.com/watch?v=1MlEv65vHgU), stated preferences and revealed preferences match in ordering or ranking between alternatives but not necessarily the magnitude. #### Asking with minimal bias Asking people about their suffering seems like a better strategy because observing is dependent on __currently__ available solutions. Our objective is to study intensity of suffering without getting biased towards whether it's efficiently and easily solvable today or not. Hence, our best bet is to directly ask people **how much worth is fixing a particular source of suffering?**. However, as we say this question has a problem of bias. If no real stakes are involved, people will answer it idiosyncratically while we want answers to be as close to truth as possible. **How can we improve upon this aspect so that answers are less biased** (and hence reducing variance, one of our desired objectives in the measure)? Well, one way we can do that is to ask people what they think _other people_ will report when asked about intensity of suffering and have a reward for the person whose answer is closest to the average of all answers. My hypothesis is that this question will make people reduce idiosyncratic bias in their responses and think carefully what generally the answer _should_ be. As the [Keynesian beauty contest](https://en.wikipedia.org/wiki/Keynesian_beauty_contest) shows, this approach reveals what the _public perception_ of intensity of suffering is rather than individual's perception of his/her suffering. This bias reduction nature of such an approach has been observed in an actual contest. >In 2011, [National Public Radio](https://en.wikipedia.org/wiki/National_Public_Radio "National Public Radio")'s _[Planet Money](https://en.wikipedia.org/wiki/Planet_Money "Planet Money")_ tested the theory by having its listeners select the cutest of three animal videos. The listeners were broken into two groups. One selected the animal they thought was cutest, and the other selected the one they thought most participants would think was the cutest. The results showed significant differences between the groups. Fifty percent of the first group selected a video with a kitten, compared to seventy-six percent of the second selecting the same kitten video. Individuals in the second group were generally able to disregard their own preferences and accurately make a decision based on the expected preferences of others. The results were considered to be consistent with Keynes' theory (via Wikipedia) In our case, the hypothetical question about intensity of suffering in depression will reveal what the perception of depression is among the people who are asked this question rather than what the individual thinks it is. However, if the group of survey takers includes people who have never had depression, the results could be hard to interpret as non-depressed people will have very little idea of how bad depression can be. So, I think for this reason **having a relatively homogeneous group of survey-takers is important** for results to be useful. #### In what units should we measure suffering? So far, we've arrived at the following question as a measure of suffering: > How much worth do you think people believe fixing this particular type of suffering is? In what units should this "worth" be measured? One simple answer would be to measure it in terms of money. That is, ask something on the lines of _"How much money do you think people will spend to fix this particular type of suffering"_. This causes the same problem we encountered before. Different people are at different economic levels and hence we'll not get reliable answers in units of money. We need a unit that all humans share more or less equally and value immensely. My proposal for such a unit is > Amount of time (in minutes, hours, days, months or years) a being is willing to reduce from their lifespan for fixing a particular type of suffering. **An important clarification**: There are usually multiple solutions for fixing the same type of suffering. In asking the question above, we're asking people to imagine a perfect solution that's instant, doesn't take any effort, doesn't cost money and is free from side effects. #### Example of the particular question to ask To people who're suffering from major depression > Imagine there's a perfect fix for depression. It's an instant fix, doesn't require any effort, doesn't cost money and is free from side effects. The way it works is this: you pop a pill that puts you into an unconscious coma and you wake up after a certain time period with your condition fixed. Assume we have a *highly depressed* patient (aged 25) who wants to try this. After getting this perfect treatment for his major depression, he's expected to live until 80 years which is another 55 years to go. According to you, what is the MAXIMUM amount of time (in minutes, hours, days, months or years) do you think this patient will be willing to go into a coma for to cure his major depression? If your answer is closest to the average of all answers I get in this survey, you'll get a $50 Amazon card. So try to think carefully and provide your best estimate. See [[Analysis of a measure of suffering from Twitter survey]] **Modification of the question for non-human sentient beings (like animals)** There are many [[Sentient beings that are capable of suffering]] beyond humans. For prioritizing [[Project ideas for reducing suffering]], it's important to have this measure for such beings as well. I propose the following phrasing of the question for non-human sentient beings. > *Set a world where humans have the same status as non-human beings have today*: Imagine a fleet of extremely intelligent and powerful aliens lands on Earth. They're so superior in their technology and co-ordination that they quickly dominate the human race. We become to them what animals are to us now: source of food, bodies for scientific experiments and cute pets. > > *Paint a situation with analogous circumstances and behavioral responses of non-human sentient beings*: In this world, imagine a specific person (let's call him Ronny). When Ronny was born, he was immediately taken away from his mother. When he's a baby, he's kept in a cage with 10 other babies (some of whom are climbing on top of him). He keeps crying for his mother all day long. The cage is in a building that's dark and windowless. Food and water comes from a tube that Ronny sucks to get fed. He shits in the cage itself and everyone's shit keeps piling on at the bottom floor of the building. As Ronny and other kids in his cage are growing up, they're so cramped for space that they're neck to neck and regularly get into fights and bite each other's nose or ears. Normally, like all humans, Ronny will live up to 80 but in this case, as soon as Ronny hits his 10th birthday, an alien robot picks him up and ties his legs to a conveyor belt. With his face down, he's taken to a bath of water that's electrified. This electric shock numbs Ronny after which his head his chopped off through an automatic cutter. > > Imagine before Ronny was born and got subjected to the life described above, his mother was offered an alternative reality where Ronny will lead an average life that a human leads now. Getting into this alternate reality doesn't require any effort, doesn't cost money and is free from side effects. The way it works is this: the new born Ronny is given pill that puts him into an unconscious coma and he wakes up after a certain time period in the alternative reality. > > Assume Ronny is an infant when he's given this treatment. After getting the treatment, in the alternative reality he's expected to live until 80 years. > > Now, what is the maximum amount of time (in minutes, hours, days, months or years) do you think Ronny's mother will be willing to have Ronny into a coma to ensure he lives the alternative life, and not the one described above? > > If your answer is closest to the average of all answers we get in this survey, you'll get a $100 Amazon card. So try to think carefully and provide your best estimate. **Modification of the question for children (who are not capable of making these decisions)** > Imagine you're a parent of a 5 year old who has gotten malaria. The kid has high fever that's not going away, and his body is full of pain and weakness. The nearby hospital is 200 kms away and you don't have a vehicle. Moreover, even if you visit there, since you don't have money to pay for the treatment, there's little you'll be able to do. Any way, there are many kids with Malaria in the hospital already so you won't get enough medical attention. > > Imagine there's a perfect fix for malaria. It's an instant fix, doesn't require any effort, doesn't cost money and is free from side effects. The way it works is this: you pop a pill that puts you into an unconscious coma and you wake up after a certain time period with your condition fixed. > > Assume your kid is 5 year old and you want to try this. After getting this perfect treatment for major depression, your kid is expected to live until 80 years which is another 75 years to go. > > Now, what is the maximum amount of time (in minutes, hours, days, months or years) do you think you will be willing to have your kid go into a coma for to cure their malaria? If your answer is closest to the average of all answers we get in this survey, you'll get a $100 Amazon card. So try to think carefully and provide your best estimate. #### Open questions There are certain #openquestions that need to be answered: ##### Can people predict the intensity of suffering? For answers from asking people to be valid, we should know that their predictions aren't systematically biased. But they usually are: - Across a wide range of health conditions (particularly somatic/body diseases), patients typically report greater happiness and quality of life than healthy people predict (if they're in similar condition) [1](https://link.springer.com/article/10.1007/s11136-015-1018-3) [2](https://core.ac.uk/download/pdf/194458381.pdf) - However, for mental diseases/disorders, patients report lower happiness and quality of life than healthy people (if they're in similar condition) [1](https://link.springer.com/article/10.1007/s11136-015-1018-3) **Hypothesis for why this might be happening**: this might be because for somatic diseases the environment can be accommodated to adjust to the condition (e.g. change of jobs, understanding partner, change of schedule, diet and so on) but for mental diseases, the unhappiness stays in the head forever. **The healthy ones err because of lack of full appreciation in their imagination**. They they're unable to fully imagine all accommodations that happen to readjust happiness in somatic case and the near permanent precense of negative thoughts in mental case. Another hypothesis could be self-reports where patients (of mental conditions) exaggerate or (of somatic conditions) underplay their happiness, but to me this seems less likely as they truly understand their conditions. General public cannot be expected to understand their condition better than themselves. ![[Screenshot 2021-07-28 at 11.53.21 AM.png]] via [Misimagining the Unimaginable: The Disability Paradox and Health Care Decision Making](https://core.ac.uk/download/pdf/194458381.pdf) ##### How does medium of asking questions affects answers? How does asking questions in person v/s online v/s on a call change the answers? ##### How do we handle worse than death suffering? From [this paper](https://onlinelibrary.wiley.com/doi/abs/10.1002/hec.1069): >The conventional time trade off (TTO) method relies on fundamentally different procedures to assess states better than and worse than dead. Arbitrary transformation mechanisms are then applied to worse than dead scores in order to achieve symmetry with those rated as better than dead. We use a ‘life profile’ approach along with a ranking procedure in order to show how states rated worse than dead may be assessed in exactly the same manner as better than dead scores. We then explore a common issue associated with states worse than dead that has received some attention recently: maximal endurable time. Our results showed that, although the severe health state was commonly rated as worse than dead, there were relatively few respondents that exhibited MET preferences. We discuss the implications of our findings for the use of the TTO method in deriving values for states that are worse than dead. via [Protocols for Time Tradeoff Valuations of Health States Worse than Dead: A Literature Review](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.880.6456&rep=rep1&type=pdf) A variation can be made where three health states are considered: full health, health condition to be evaluated and death. Two two scenarios can be presented, one with full health and the other one with poor health. Both scenarios are followed by death. An additional lead time of full health is added so that there is more time to trade off in case people have health states worse than death. ![[Screenshot 2021-07-28 at 12.52.34 PM.png]] ![[Screenshot 2021-07-28 at 12.52.57 PM.png]] Although [this study](https://www.sciencedirect.com/science/article/pii/S1098301518361953) found out that the method above (adding a lead time to measure worse than death cases) is not discriminative enough. >From a purely information theoretical perspective, this indicates that there is no practical gain in the collection and analysis of negative values; they appear to be uninformative as to the relative severity of health states I think that makes sense because **death is the ultimate cessation and states worse than it perhaps are impossible to differentiate**. ##### How should we interpret the answers? Should be in units of time or units of % of remaining lifespan? If we use TTO where we ask 10 years of perfect health, it should be in terms of years only. (For calculating [QALY](https://en.wikipedia.org/wiki/Quality-adjusted_life_year), it can trivially be scaled between 0 and 1, where 1 means perfect health and 0 means death ) ##### How should we compare two sufferings of different creatures? ##### How does the age of the respondent biases the answer? It seems people automatically assume the answer to be valid relative to reference group of their own age. [1](https://core.ac.uk/download/pdf/194458381.pdf) So, it may help to clarify like assume perfect health for your age group or perfect health for a 20 year old. These two answers will be different. ##### How does conversational context impact answers? If you set the context as being a survey among people with a specific disease, people are likely to implicitly report their well being relative to that disease. However, if you set the context as general population, they will report their well being relative to population. [1](https://core.ac.uk/download/pdf/194458381.pdf) Context matter so it's best to set it clearly. ##### How do multiple sources of suffering confound answers? >when people experience multiple disabilities, their well-being declines signifi-cantly (Mehnert, Krauss, Nadler, & Boyd, 1990). ##### How does incidence of that particular type of suffering in age influences this question (e.g. Alzheimer's happens only in old age or infant death syndrome only happens for infants) As per [this paper](https://www.tandfonline.com/doi/full/10.1080/14737167.2020.1779062), responses are distorted by time preferences and respondents life expectancies. There are ethical issues in asking terminally ill people about how much they value remaining life or giving hopes of 10 more years when they don't exist Perhaps, discrete choice analysis is a better method for assessing those. See for example a quality of life measure for infants: [A two-step procedure to generate utilities for the Infant health-related Quality of life Instrument (IQI)](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0230852) >Several studies discourage the use of conventional economic valuation methods, such as the time trade-off, with this age group because caregivers (proxies) are apparently unwilling to trade off a child’s life years, leading to relatively high values for poor health states (from [here](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0230852)) ##### What does suicide tell us about this question? ##### What's the cheapest way to test this measure against objectives (mentioned in the document) Did a survey on twitter. See [[Analysis of a measure of suffering from Twitter survey]]. Worked out reasonably well as answers tallied with what was found in literature. ##### How does this work for non-adult-human sentient beings (like children or chicken)? ##### How does asking people to guess average answer v/s giving a direct answer impact responses? #openquestion This study probably hasn't been done, so should be interesting to do. It'll be a novel contribution to the area of TTO research. #### Limitations of Time Trade Off method via [An overview of the time trade-off method: concept, foundation, and the evaluation of distorting factors in putting a value on health](https://www.tandfonline.com/doi/full/10.1080/14737167.2020.1779062) >The TTO is confounded by **time preferences and by respondents’ life expectancies**. TTO is cognitively challenging, therefore guidance during the interviews is needed, producing interview effects. TTO does not measure one thing at a time, nor are the values independent of other states that are being valued in the same task. #### Lessons: - The Time Trade Off ([see review here](https://link.springer.com/content/pdf/10.1007/s10198-013-0508-x.pdf) or [standardized EuroQol methodology](https://www.ohe.org/news/new-publication-euroqol-protocols-time-trade-valuation-health-states) or a similar one) method presents two scenarios which keep on getting varied until a point of indifference is reached between then (e.g. 10 years with condition v/s 9 years of health, 8 years of health and so on..). Protocol - Add a warm up excercise to ask about their state of health (perhaps ask EQ-5D questionaire) - Scenario 1: 10 years of full health, followed by death; Scenario 2: 10 years of health with condition X. Ask people to choose either scenario 1, scenario 2 or being indifferent to either one. - If they choose 10 years of health with condition, they probably haven't understood the survey properly - Then ask them between immediate death v/s 10 years in diseased state - If they choose immediate death, you have worse than death scenario. Stop there. - On a visual scale, have them keep reducing timespan of perfect health by a minimum unit until they reach a point of indifference between scenario 1 and 2. - Record the point of indifference - If even at 0 years of full health (which is immediate death), they prefer scenario 1 v/s scenario 2, record this result as Worse than Death - Record methodology using [this checklist](https://link.springer.com/content/pdf/10.1007/s10198-013-0508-x.pdf) - Use a visual aid like the following - ![[8 June - TTO figure.jpg]] - [A sample questionnaire](https://www.york.ac.uk/che/pdf/op20.pdf) - Survey minimum 100 people per health state ([see](https://link.springer.com/content/pdf/10.1007/s10198-013-0508-x.pdf)) - Exclude people who didn't seem to understand questions or didn't spend enough effort/time - Survey people suffering with the condition _right now_, their answers are better representation of true suffering as people without that condition don't imagine well or misremember real conditions. - If you're asking people to imagine a different state (because perhaps collecting data is difficult, say someone in ICU or impossible, say caged chicken), then it's important to describe their situation as fully and as objectively as possible - Analyze results by different age groups or condition variants (in order to keep reference group as similar as possible during interpretation) #### Reading material - [Measuring Animal Welfare: Philosophical foundations, practical indicators and overall assessments](https://c2b5df1e-0ba2-4201-9fb6-87e92e6ad2c0.usrfiles.com/ugd/c2b5df_b7873f253e2a4a4ab129d0252c7e5245.pdf) > In conclusion, all of the welfare indicators and overall assessments of welfare reviewed in this report have numerous flaws. Establishing a valid objective assessment for the welfare of individuals is a very difficult task and no single system provides the “silver bullet” for measuring welfare. The welfare assessments of every system need to be held in the context of each system's flaws and methods of calculation, and interpreted with this in mind. Instead, a better method for assessing welfare would be a composite of multiple qualitative and quantitative methods which each provide a different perspective on welfare. If multiple systems and methods of assessments converge on which welfare improvements we should prioritise, we can have greater confidence than relying on any given m e a s u r e . T h e i d e a l c o m b i n a t i o n o f measures would be some combination of a qualitative measure, expert opinion based measures, an index or semantic model of animal-based measures and standalone measures, such as preference testing or qualitative behavioural assessment. This could be the combined use of the Five Domains, a semantic model of expert opinion, a SOWEL-type model and any standalone measures, such as preference t e s t i n g o r q u a l i t a t i v e b e h a v i o u r a l assessment. In practice, the cost of using such an extensive list will make it impractical for many decision-makers, but this could be the basis for large-scale asks. A more limited combination would be the u s e o f t h e F i v e D o m a i n s , C h a r i t y Entrepreneurship Welfare Index, any standalone measures and interviews with experts - [Welfare Adjusted Life Years](https://www.wellbeingintlstudiesrepository.org/cgi/viewcontent.cgi?article=1007&context=assawel) - [Disability Adjusted Life Years](https://www.who.int/quantifying_ehimpacts/publications/en/9241546204chap3.pdf) #inbox