eupolicy.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
This Mastodon server is a friendly and respectful discussion space for people working in areas related to EU policy. When you request to create an account, please tell us something about you.

Server stats:

226
active users

#arxiv

3 posts3 participants0 posts today
Continued thread

The extended argument, why it is the wrong hammer for physics nails, but good for engineering questions:
arxiv.org/abs/2405.18095#ai4sc #arxiv

arXiv logo
arXiv.orgIs machine learning good or bad for the natural sciences?Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology - in which only the data exist - and a strong epistemology - in which a model is considered good if it performs well on held-out training data. These philosophies are in strong conflict with both standard practices and key philosophies in the natural sciences. Here we identify some locations for ML in the natural sciences at which the ontology and epistemology are valuable. For example, when an expressive machine learning model is used in a causal inference to represent the effects of confounders, such as foregrounds, backgrounds, or instrument calibration parameters, the model capacity and loose philosophy of ML can make the results more trustworthy. We also show that there are contexts in which the introduction of ML introduces strong, unwanted statistical biases. For one, when ML models are used to emulate physical (or first-principles) simulations, they amplify confirmation biases. For another, when expressive regressions are used to label datasets, those labels cannot be used in downstream joint or ensemble analyses without taking on uncontrolled biases. The question in the title is being asked of all of the natural sciences; that is, we are calling on the scientific communities to take a step back and consider the role and value of ML in their fields; the (partial) answers we give here come from the particular perspective of physics.

Unsichtbare Manipulation im Peer-Review? Forscher platzieren versteckte KI-Prompts in wissenschaftlichen Papieren, um Review-Systeme zu beeinflussen. Eine Analyse auf arXiv zeigt, wie angreifbar Forschung durch generative KI geworden ist. Was bedeutet das für die Wissenschaft? #arXiv #OpenScience #KI 👇
all-ai.de/news/topbeitraege/wi

All-AI.deSkandal um versteckte KI-Prompts erschüttert Peer-Review-VerfahrenEine Nikkei-Analyse deckt systematische Manipulation auf – ist die wissenschaftliche Integrität in Gefahr?

LLMs should not replace therapists

arxiv.org/abs/2504.18412

arXiv logo
arXiv.orgExpressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providersShould a large language model (LLM) be used as a therapist? In this paper, we investigate the use of LLMs to *replace* mental health providers, a use case promoted in the tech startup and research space. We conduct a mapping review of therapy guides used by major medical institutions to identify crucial aspects of therapeutic relationships, such as the importance of a therapeutic alliance between therapist and client. We then assess the ability of LLMs to reproduce and adhere to these aspects of therapeutic relationships by conducting several experiments investigating the responses of current LLMs, such as `gpt-4o`. Contrary to best practices in the medical community, LLMs 1) express stigma toward those with mental health conditions and 2) respond inappropriately to certain common (and critical) conditions in naturalistic therapy settings -- e.g., LLMs encourage clients' delusional thinking, likely due to their sycophancy. This occurs even with larger and newer LLMs, indicating that current safety practices may not address these gaps. Furthermore, we note foundational and practical barriers to the adoption of LLMs as therapists, such as that a therapeutic alliance requires human characteristics (e.g., identity and stakes). For these reasons, we conclude that LLMs should not replace therapists, and we discuss alternative roles for LLMs in clinical therapy.

I did a postdoc and some lecturing at KAIST, the Korea Advanced Institute of Science and Technology, and really liked it there -- so I'm especially disappointed to hear about arXiv postings that include hidden AI prompts to generate better reviews of the paper:

asia.nikkei.com/Business/Techn

(This sort of hidden text stuff was a big deal about 20 years ago, in the early, wilder era of SEO, but Google and the other search engines got wise to it -- if they mostly solved that problem so long ago, why is it that modern AI web-crawling stuff is vulnerable to this? It seems super simple for them to have their AI crawler bots or agents to access web pages in a way that filters or otherwise accounts for hidden or very very small text.)

Nikkei Asia · 'Positive review only': Researchers hide AI prompts in papersBy Staff Writer

Important post on building a (dark, offline) mirror of Arxiv, by TIB, the Technische Informationsbibliothek, run by the Leibniz Information Centre for Science and Technology, in Hannover, Germany. The blog also explains the license / copyright challenges. And has some 🔥:

> “So when the Trump administration makes decisions that have fatal consequences for science and research in the US, the repercussions reach far beyond the Gulf of Mexico”

blog.tib.eu/2025/05/14/protect

TIB-Blog · Protecting Science: TIB builds Dark Archive for arXiv - TIB-Blog
More from Esther Tobschall