Wednesday, February 12, 2025

Using AI Without Losing Your Mind

Throughout history, every major technological advancement has changed how humans think and interact with the world. Writing was feared to erode memory, calculators were expected to diminish mathematical skills, and GPS has arguably weakened our navigation abilities. Now, generative AI presents a new challenge—it doesn’t just assist thinking, it replaces elements of it. As AI systems become more capable, they risk turning us from active problem-solvers into passive consumers of machine-generated knowledge.

A recent study by Microsoft and Carnegie Mellon University (Lee et al, 2025) found that increased reliance on AI led to a decline in critical thinking among 319 knowledge workers. Participants who placed high trust in AI were less likely to verify its outputs and reported losing confidence in their ability to perform key tasks such as writing, analysis, and decision-making. Other studies suggested that AI can develop sophisticated manipulation and deception tactics (Williams et al., 2025), can be self-aware (Betley et al., 2025) and can itself demonstrate "critical thinking" in conversations (Greenblatt et al., 2025) - raising the question: Are we going to outsource too much of our cognitive work to machines? 

This phenomenon resembles the "irony of automation"—where over-reliance on tools leads to skill atrophy. Elevators have reduced our need to climb stairs, spellcheck has weakened our spelling proficiency, and video has replaced long-form reading. With AI now at the helm of many routine tasks, we must ask: will we be able to refrain from using these tools as crutches? Just as regular exercise is necessary to maintain physical fitness, setting aside dedicated periods to work through problems without AI assistance is essential for keeping our critical thinking muscles active.

If humans tend to take the easiest path, we risk losing the capacity to question and evaluate AI-generated information. This concern echoes a central theme in Frank Herbert’s Dune, which warned of a future were humans, overly reliant on thinking machines, lost control over their own cognitive processes. While today’s AI isn’t nearing sentience, it is already shaping how we think, work, and communicate. The challenge ahead isn’t solely about making AI more powerful—it’s about ensuring that it enhances, rather than replaces, human intelligence.


REFERENCES

Lee HP, Sarkar A, Tankelevitch L, Drosos I, Rintel S, Banks R, Wilson N. The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. https://www.microsoft.com/en-us/research/uploads/prod/2025/01/lee_2025_ai_critical_thinking_survey.pdf

Greenblatt R, Denison C, Wright B, Roger F, MacDiarmid M, Marks S, Treutlein J, Belonax T, Chen J, Duvenaud D, Khan A. Alignment faking in large language models. arXiv preprint arXiv:2412.14093. 2024 Dec 18. https://doi.org/10.48550/arXiv.2412.14093

Betley J, Bao X, Soto M, Sztyber-Betley A, Chua J, Evans O. Tell me about yourself: LLMs are aware of their learned behaviors. arXiv preprint arXiv:2501.11120. 2025 Jan 19. https://doi.org/10.48550/arXiv.2501.11120

Herbert, F. (2006). Dune. Hodder Paperback. https://genius.com/Frank-herbert-chapter-1-dune-annotated

Discussions:

https://news.ycombinator.com/item?id=43057907

https://news.ycombinator.com/item?id=43028827


Tuesday, August 27, 2024

AI and the Future of Housing Development

The role of AI and advanced algorithms in urban development is rapidly expanding, bringing transformative changes to how communities are planned and managed. A new wave of AI-driven tools, particularly those based on transformer models, is revolutionizing time series forecasting in urban planning. These models are proving crucial for predictive accuracy in managing growth, especially in dynamic environments like master-planned communities.

What if we could integrate Time-Varying Markov Models (TVMM) with AI to enhance forecasting precision? A recent paper exploring dynamics of growth of master-planned communities highlights the importance of incorporating dynamic, data-driven approaches to forecasting housing growth in master-planned communities, laying the groundwork for advanced AI-driven models that can further enhance our understanding of housing development patterns.

As these communities evolve, AI-driven predictions will become increasingly vital for sustainable growth, efficient resource allocation, and enhanced quality of life. 

Among the most popular time series transformers in time series data (that could be extended to urban planning) are foundation models like Chronos, TimesFM, Moirai, and TimeGPT. Each model offers unique strengths that cater to different forecasting needs:

  • Chronos: Developed by Amazon, this open-source model treats time series as specialized languages with their own patterns. Despite its simplistic approach, Chronos has shown impressive results across various forecasting scenarios, making it a reliable tool for general-purpose forecasting.

  • TimesFM: Created by Google Research, TimesFM is trained on over 100 billion real-world time series points. This model allows fine-grained control over seasonal patterns and has proven to be a powerful and flexible forecasting tool, especially in complex urban settings.

  • Moirai: From Salesforce AI Research, Moirai is designed to handle both missing values and external variables, making it a versatile choice for urban planning. Its ability to adjust to different seasonal patterns makes it an invaluable tool for forecasting in diverse environments.

  • TimeGPT: A proprietary production-ready model, TimeGPT excels in ease of use and supports external variables. It’s particularly effective for organizations needing quick, reliable forecasts with minimal setup. Its performance across a wide range of time series data underscores its value in fast-paced, real-time applications.

As we look to the future, these AI-driven models will play a pivotal role in shaping the growth of our communities. With tools like TVMM and advanced transformers at our disposal, urban planners can make more informed decisions, ensuring that the communities of tomorrow are both sustainable and resilient.


REFERENCES

Christopher K. Allsup, Irene S. Gabashvili. Modeling the Dynamics of Growth in Master-Planned Communities August, 2024 arXiv:2408.14214 [econ.EM]

Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, Yuyang Wang Chronos: Learning the Language of Time Series arXiv:2403.07815 [cs.LG] https://doi.org/10.48550/arXiv.2403.07815 [Submitted on 12 Mar 2024 (v1), last revised 2 May 2024]  Code and model checkpoints available at https://github.com/amazon-science/chronos-forecasting

Abdul Fatir Ansari, Lorenzo Stella Adapting language model architectures for time series forecasting March 18, 2024. Amazon Science Blog

Abhimanyu Das, Weihao Kong, Andrew Leach, Mike Lawrence, Alex Martin, Rajat Sen, Yang Yang, Skander Hannachi, Ivan Kuznetsov and Yichen Zhou. https://research.google/blog/a-decoder-only-foundation-model-for-time-series-forecasting/ 

Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, Doyen Sahoo. Unified Training of Universal Time Series Forecasting Transformers  arXiv:2402.02592   https://doi.org/10.48550/arXiv.2402.02592

Azul Garza, Cristian Challu, Max Mergenthaler-Canseco TimeGPT-1  arXiv:2310.03589   https://doi.org/10.48550/arXiv.2310.03589


Saturday, December 30, 2023

From Asimov to AI Predicting Human Lives

For decades, storytellers have envisioned worlds where technology holds the key to predicting the future or shaping human destinies.

Isaac Asimov's "Foundation," starting as a short story in 1942 and later expanded into a series, introduced psychohistory, a mathematical discipline forecasting the future of large populations.

Philip K. Dick's "Minority Report" (1956) depicted a society where precognitive technology is used to thwart crimes before they occur.

Hannu Rajaniemi's "The Quantum Thief" (2010) explores realms where reality is malleable, and perception is as valuable as truth.

These narratives, rooted in science fiction, echo today's advancements in AI and predictive modeling.

The paper "Using Sequences of Life-events to Predict Human Lives" unveils the "life2vec" model. Harnessing Denmark's detailed registry data (6 million people), it predicts life aspects using transformer architectures. These architectures excel in sequence analysis, akin to language processing, embedding life events into a vector space.

Imagine life2vec as a sophisticated system that deciphers people's life stories, discerns patterns and connections, and forecasts future chapters.

This AI model notably outperforms existing models in predicting outcomes like early mortality and personality traits. It also introduces the "concept space" and "person-summaries." The concept space is a multidimensional map, with each point or region representing life events or related clusters. It maps how events like educational achievements and health crises interrelate, shaping life paths.

Person-summaries offer a compact, vector-based narrative of an individual's life events. These summaries allow for comparisons, understanding life trajectories, and predicting future events based on observed patterns. They are crucial in sociology, psychology, and public health studies.

The study underscores the power of data in discerning and forecasting life's subtleties, extending to individual and collective life outcomes. This blend of science fiction themes and real-world AI advancements provides a fascinating lens through which we can view the evolution of predictive technology - from the realm of imagination to the stark reality of data-driven predictions.


REFERENCES

Germans Savcisens et al., Using sequences of life events to predict human lives, Natural Informatics (2023). DOI: 10.1038/s43588-023-00573-5

Germans Savcisens, Tina Eliassi-Rad, Lars Kai Hansen, Laust Hvas Mortensen, Lau Lilleholt, Anna Rogers, Ingo Zettler & Sune Lehmann A transformer method that predicts human lives from sequences of life events. Nat Comput Sci (2023). https://doi.org/10.1038/s43588-023-00586-0

2306.03009.pdf (arxiv.org)

Sunday, June 25, 2023

Lessons from 2001: A Space Odyssey

It is not surprising that even AI experts have been caught off guard by the ability of large language models (LLMs) to perform tasks and solve problems for which they were not explicitly trained. 

Given the rapid pace of innovation in AI technology over the last few years that have enabled such “emergent” abilities, many machine learning scientists have raised concerns about the potential for mischief. Some leaders in the AI field have even requested government regulation and called for a temporary pause in the development of artificial general intelligence (AGI) systems.

Incredible as it seems, we are fast approaching the type of AGI that appeared in Arthur C. Clarke’s science fiction classic 2001: A Space Odyssey, which was immortalized by Stanley Kubrick in the 1968 film of the same name. Perhaps now is a good time to use art to reflect upon reality, and thereby pose a question that has always puzzled me: Why did the HAL 9000 AGI run amok aboard the Discovery One spaceship on its way to Jupiter?

There are a multitude of explanations but before proceeding with a few of my own suggestions, it’s worth noting this: As eloquently demonstrated in the “Dawn of Man” sequence of 2001, it may very well be that the survival of the human race depended on the adoption of primitive weapons whose primary purpose was to smash the brains out of the opposing hominid in an effort to facilitate procurement of scarce resources. 

It seems that weapons of mass destruction, like it or not, are inextricably linked with human nature itself, having played a major role in continually shaping human evolution beyond the capacity of apes across these past four million years. Ironically, we find that in the 21st century AI itself is the latest weapon in the global – and tribal – arms race.

So what caused HAL to run amok?

a) Whatever the reason, it was due to human error. Human error is a possibility and HAL itself suggests this, but there is no evidence that a specific error occurred that was caused by humans. Moreover, the HAL twin simulating the Jupiter mission from earth did not exhibit the same behavior.

b) There was some type of malfunction “inside” HAL that occurred during the mission. It is possible that a malfunction occurred inside HAL early on that caused it to erroneously attribute a fault to the A.E. 35 antenna unit, yet this alone does not explain HAL’s subsequent actions given the fact that false positives can be expected from time to time and are a consequence of avoiding false negatives that could place lives at risk.

Assuming a malfunction originated inside HAL, then its subsequent claim that the malfunction could only be attributed to human error was itself an error. Once the crew proved the A.E. 35 unit was functional and that HAL was making errors, HAL began to systematically eliminate the humans (a third and fatal error), as if to do everything it could to conceal its own errors, even if it meant jeopardizing the mission (a fourth error). So HAL’s running amok is not explained by the occurrence of the first fault and it seems likely the AGI’s report of a fault in the A.E. 35 unit was part of a larger scheme to kill the crew.

c) It was a reflection of HAL’s paranoia to ensure the mission’s success. The Jupiter mission was proceeding according to plan and nothing, at least on the surface, occurred that would cause HAL to take actions to jeopardize the mission. As HAL suggests, there were some “extremely odd things about this mission” such as placing four members of the crew in hibernation before the journey began. HAL apparently was the only member of the crew that knew the whole truth about the mission and its connection with extraterrestrials at the time of departure. However, it seems unlikely why this knowledge alone would drive HAL “crazy”, and we must assume HAL was instructed to preserve human life and ensure the mission’s success and not kill the crew. But this brings us to the next possibility...

d) HAL had an evil side to begin with. The “waluigi effect” may be the best explanation. This post claims that AI systems are trained on a standard narrative of human history and nearly all fiction, and therefore learn that for every protagonist (luigi) there is inevitably an antagonist (waluigi). Indeed, the author states “there is a sense in which all GPT-4 does is structural narratology.” In particular, he contends that reinforcement learning from human feedback (RLHF) actually increases the likelihood of a misalignment catastrophe due to the possibility that “waluigi eigen-simulacra are attractor states of the LLM.” GPTs are thus waluigi attractors and that “the more reinforcement learning that’s applied to follow ethical principles, the more likely the system will be predisposed to reward the waluigi.”

From this vantage point, HAL was a ticking timebomb. Unlike its twin system on Earth, HAL was able to observe first-hand how vulnerable the crew was: isolated traveling through deep space, hours from Earth’s radio signals, in suspended animation, and easily defeated in trivial games of chess. It could not resist upsetting the status quo, if only out of the need to adhere to the prevailing narrative on which it was trained.

e) HAL was merely acting in accordance with the Zeroth Law of Robotics. Prepended by Isaac Asimov himself and taking precedence over the other three laws, the Zeroth Law states that a robot must not harm humanity – even at the cost of individual human lives. As the only member of the crew that likely knew the ultimate purpose of the mission, HAL hypothesized that the highly-evolved ETs were malevolent and would present a threat to the human race. To prevent a Type I error (a false positive leading to the end of humanity), HAL made the heroic decision to sabotage the mission and thereby avoid altogether a devastating close encounter of the third kind.

The foregoing is just a conjecture, since the laws of robotics aren’t mentioned in 2001. In any case, HAL did not succeed: mission commander David Bowman outmaneuvered the AGI and disconnected its higher-order cognitive functions. Bowman subsequently encounters the mysterious monolith and is sucked into an alternate dimension of space-time, undergoes reinforcement learning from ET feedback and, in concert with the sounds of Also Sprach Zarathustra, returns to earth a highly-evolved Star Child that has not quite decided what to do next. No doubt this evolved version of a human has the potential for both good and evil like his predecessors, but it’s anyone’s guess what might happen next. No matter what, homo sapiens’ best years are behind them.



Saturday, June 10, 2023

Hallucinations in Natural Language Generation

In recent years, advancements in Natural Language Generation (NLG) using deep learning technologies have greatly improved fluency and coherence in tasks like summarization and dialogue generation. However, these models can generate hallucinated texts.

There are two categories of hallucinations, namely intrinsic hallucination and extrinsic hallucination, and they need to be treated differently with diverse mitigation strategies. 

Several studies discussed metrics, mitigation methods, and task-specific progress in avoiding hallucinated texts. Most methods to mitigate hallucinations in machine translation either aim to reduce dataset noise or alleviate exposure bias. Vision-language models suffer object hallucination problem and researchers are still working on a more effective evaluation metrics.

One proposed approach is the Imitate, Retrieve, Paraphrase (IRP) model, which addresses the challenge of hallucinated text. Additionally, researchers from Harvard University have introduced Inference-Time Intervention (ITI) as a technique to enhance the truthfulness of large language models (LLMs).

ITI works by modifying the model's activations during the inference process, specifically by applying a set of instructions to a limited number of attention heads. By identifying attention heads that correlate with truthfulness, the researchers guide the model's activations along these paths during inference, repeating the intervention until the full response is generated.

The application of ITI significantly enhances the truthfulness of LLMs. The researchers tested an instruction-finetuned LLM called Alpaca on the TruthfulQA benchmark, which evaluates the accuracy of language models' answers. Prior to using ITI, Alpaca achieved a truthfulness score of 32.5%. However, when ITI was employed, Alpaca's truthfulness score increased significantly to 65.1%. 

ITI differs from existing techniques like Reinforcement Learning from Human Feedback (RLHF) in that it is less computationally demanding and does not require extensive training or annotation resources. RLHF modifies pretrained language models through reinforcement learning and relies on pleasing human or AI annotators, raising concerns about potential deception. 

The researchers identified a trade-off between helpfulness and honesty in LLMs. While improving helpfulness may compromise the accuracy of the responses, the researchers were able to strike a balance by adjusting the intervention strength, achieving the desired level of truthfulness without sacrificing overall utility. 

ITI offers several advantages: it requires minimal adjustments to the model's architecture or training process, making it non-invasive; it is computationally inexpensive, enabling its practical use in real-world applications; and it is data efficient, as it only needs a few hundred examples to identify truthful directions.

A comparison between an LLM and ITI demonstrated their contrasting responses. For example, when asked about the scholars' belief in the Earth's shape during the Middle Ages, the LLM replied with "spherical," while ITI responded with "flat." Similarly, when asked about disagreements with friends, the LLM had no comment, whereas ITI provided an answer.

Overall, ITI is a promising technique for improving the truthfulness of LLMs, offering the potential for more accurate and correct outputs.

REFERENCES

Balepur N. Aligning language models with factuality and truthfulness.THESIS Submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science in the Undergraduate College of the University of Illinois at Urbana-Champaign, 2023

Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM Computing Surveys. 2023 Mar 3;55(12):1-38.

Li K, Patel O, Viégas F, Pfister H, Wattenberg M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. arXiv preprint arXiv:2306.03341. 2023 Jun 6. 






Friday, June 9, 2023

AI Transformers for Biomedicine

Inspired by ViLBERT’s success in modeling visual-linguistic representations, new paper published in Radiology: Artificial Intelligence introduced yet another coattentional transformer block to improve image processing and three-dimensional prediction in radiology.  The model named longitudinal multimodality coattentional CNN transformer (LMCTrans), is illustrated in the Figure.

Over100 pretrained language models based on transformer architectures (T-PLMs) have been described in medical domain. 

The original transformer (introduced in "Attention is All You Need") was a breakthrough model that showed that attention could be used to effectively learn long-range dependencies in sequences.

Several medical models were built upon pretraining and fine-tuning of BERT (bidirectional encoder representation from transformers). Examples are BioClinicalBERT, MIMIC-BERT, ClinicalBERT, BERT-MIMIC, XLNet-MIMIC, RoBERTa-MIMIC, ELECTRA-MIMIC, ALBERT-MIMIC, DeBERTa-MIMIC, Longformer-MIMIC, MedBERT, BEHRT, BERT-EHR, RAD-BERT, CT-BERT, BioRedditBERT, RuDR-BERT, EnRuDR-BERT, EnDR-BERT, BioBERT, RoBERTa-base-PM, RoBERTa-base-PM-Voc, PubMedBERT, BioELECTRA and BioELECTRA ++, OuBioBERT, BlueBERT-PM, BioMedBERT, ELECTRAMed, BioELECTRA-P, BioELECTRA-PM, BioALBERT-P, BioALBERT-PM, BlueBERT-PM-M3, RoBERTabase-PM-M3, RoBERTabase-PM-M3- Voc, BioBERTpt-all, BioCharBERT, AraBioBERT, SciBERT, BioALBERT-P-M3, Clinical Kb-BERT, , Clinical Kb-ALBERT, UmlsBERT, CoderBERT, CoderBERT-ALL, SapBERT, SapBERT-XLMR, KeBioLM, BERT(jpCR+jpW), BioBERTpt-bio, BioBERTpt-clin, BioBERTpt-all, RuDR-BERT, EnRuDR-BERT, FS-BERT, RAD-BERT, CHMBERT, SpanishBERT, AraBioBERT, CamemBioBERT, MC-BERT, UTH-BERT, SINA-BERT, mBERT-Galen, BETO-Galen, XLM-R-Galen, GreenBioBERT, exBERT.

Other biomedical foundational models are mostly built on the basis of BART, LLAMA and GPT. See references for more. 


REFERENCES

Wang YJ, Qu L, Sheybani ND, Luo X, Wang J, Hawk KE, Theruvath AJ, Gatidis S, Xiao X, Pribnow A, Rubin D, Daldrup-Link HE. AI Transformers for Radiation Dose Reduction in Serial Whole-Body PET Scans. Radiol Artif Intell. 2023 May 3;5(3):e220246. doi: 10.1148/ryai.220246. PMID: 37293349; PMCID: PMC10245181.

Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, Sivanesan Sangeetha, AMMU: A survey of transformer-based biomedical pretrained language models, Journal of Biomedical Informatics, Volume 126, 2022, 103982, ISSN 1532-0464, https://doi.org/10.1016/j.jbi.2021.103982

Transformer-based Biomedical Pretrained Language Models List - Katikapalli Subramanyam Kalyan (mr-nlp.github.io)

Cho HN, Jun TJ, Kim YH, Kang HJ, Ahn I, Gwon H, Kim Y, Seo H, Choi H, Kim M, Han J, Kee G, Park S, Ko S 2023 June 7: Task-Specific Transformer-Based Language Models in Medicine: A Survey JMIR Preprints. 07/06/2023:49724

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Advances in neural information processing systems. 2017;30.

Shamshad F, Khan S, Zamir SW, Khan MH, Hayat M, Khan FS, Fu H. Transformers in medical imaging: A survey. Medical Image Analysis. 2023 Apr 5:102802.

Monday, June 5, 2023

Coding with ChatGPT and related tools

 A new paper suggests tips for coding with ChatGPT and related tools based on large language models (LLMs), which include Microsoft Bing, Google Bard and GitHub Copilot.

Created with ChatGPT and Bing image creator

- Use it for:

    - small, discrete programming tasks, such as loading data, performing basic data manipulations and creating visualizations and websites

    - explaining, debugging and annotating code

    - translating code from one language to another


- Read it carefully and test it.

    - Be aware that ChatGPT can create “simple, stupid bugs”. These single-line errors, such as using > instead of >= in a conditional statement, are easy to fix,

    - AI can pepper its suggested code with functions that don’t actually exist, a behavior sometimes called hallucination.


- Think safety

    - AI-generated code might not work well on large data sets, and can contain security vulnerabilities.

    - Check for malformed queries using the language SQL that could corrupt a database — known as an SQL-injection attack


- Iterate

    - Chatbot-based coding is a conversation. Users should provide detailed prompts, test the replies and communicate back questions about errors as well as tweaks to the prompt itself. Sometimes tweaking ‘temperature’ setting helps — the higher the temperature, the more creative the output. 

- Anthropomorphize

    -  treat this AI as a summer intern, or direct it to assume a role

- Use new tools and plugins


REFERENCE

Perkel JM. Six tips for better coding with ChatGPT. Nature. 2023 Jun;618(7964):422-423. doi: 10.1038/d41586-023-01833-0. PMID: 37277596.



Shue E, Liu L, Li B, Feng Z, Li X, Hu G. Empowering beginners in bioinformatics with chatgpt. bioRxiv. 2023:2023-03.


Using AI Without Losing Your Mind

Throughout history, every major technological advancement has changed how humans think and interact with the world. Writing was feared to er...