Foundation Models Errors: Designing a guardrailViolation Fallback

Apple's on-device model rejects input caught by its safety filter. That filter also caught an ordinary Korean sentence.

When it rejects a request, there is no summary. If the screen is empty, the user thinks the app is broken, even though the text they sent is still there.

What to show instead when it fails, and how to record that failure so it can be fixed later: this article covers both. In production, using an on-device model means the failure path contains more code than the success path.

Splitting errors by string fails quietly

At first I scanned the error description as a string, looking for words such as guardrail, safety, and refus.

This is the text Apple gives for a safety-filter rejection.

"May contain sensitive content"

None of the words I searched for appears. The safety rejection was not classified and fell through as “unknown error.”

The screen showed “Something went wrong while processing · Try again.” The user tapped it. The same input was rejected again. Every tap ran the model from the beginning, and the real device became hot.

I classify by type

I inspect the cases of LanguageModelSession.GenerationError directly.

// FoundationProfileModelExecutor.swift:126-136
switch generation {
case .guardrailViolation: return .rejected("guardrailViolation")
case .refusal: return .rejected("refusal")
case .rateLimited, .concurrentRequests: return .transient("rateLimited")
case .assetsUnavailable: return .unavailable
case .exceededContextWindowSize: return .permanent("exceededContextWindowSize")
case .unsupportedGuide: return .permanent("unsupportedGuide")
case .unsupportedLanguageOrLocale: return .permanent("unsupportedLanguageOrLocale")
case .decodingFailure: return .permanent("decodingFailure")
@unknown default: return .permanent("unknownGenerationError")
}

Apple can change the wording while the case remains. @unknown default also keeps compilation intact if a new case is added.

I fold eight cases into four.

Classification	Meaning	Retry button
rejected	This input was rejected	Do not show it
transient	Try again later	Show it
unavailable	It cannot run on this device now	Do not show it
permanent	Trying again will not help	Do not show it

These four branches are the basis for retry policy. If I collapse them into one, every failure gets a retry button.

Strings crossing the boundary are machine tokens

The string in rejected("guardrailViolation") is not text for the user. It is a stable name that lets the upper layer branch on the reason.

If I send an error description containing the original text, it remains in the receipt. The user's writing leaks into logs and storage.

// FoundationProfileModelExecutor.swift:104-109
static func signature(of error: Error) -> String {
  let described = String(describing: error)
  let head = described.prefix { $0 != "(" && $0 != ":" && !$0.isNewline }
  return head.trimmingCharacters(in: .whitespaces)
}

I cut at an opening parenthesis or colon because Foundation Models' error description includes the prompt. Diagnostics need to know which failure occurred, not what the user wrote.

I store three origins for each chunk

I must not remove a failed chunk from the result. If one of seven chunks fails and I store six, the user believes the document had six chunks.

So I store how each chunk was made.

// SummaryEngine.swift:11-18
public enum SummaryOrigin: String, Codable, Hashable, Sendable {
  /// 모델이 답했다.
  case model
  /// 모델이 세 번 다 쓸 만한 답을 내지 못해, 원문에서 문장을 골라 세웠다.
  case extracted
  /// 어느 방법으로도 만들지 못했다.
  case none
}

The comments explain why this enum exists. A failed unit simply disappeared from the result array, and the document was marked complete with the remainder. To the consumer, a seven-page document looked like one chunk, with no way to know what was lost.

That amounted to saying a thing was summarized when it was not.

Origin	What appears on screen
model	Summary
extracted	Sentences selected from the source
none	Marker saying this part could not be summarized

The third state is what makes the screen honest.

I keep rejection history even for successful chunks

SummaryPiece has rejections, the reasons for answers rejected while making this chunk.

// SummaryEngine.swift:26-31
/// 이 조각을 만들며 되돌려보낸 답들의 이유. 순서는 시도 순서다.
///
/// 진단을 위해 남긴다. `origin == .model`인데 이 목록이 비어 있지 않다면 첫 답이
/// 거부됐고 다음 시도가 통과했다는 뜻이다. 프롬프트가 몇 번째에 먹히는지
/// 아는 유일한 방법이고, 그것 없이는 프롬프트를 고칠 근거가 없다.
public let rejections: [String]

It is easy to think that success means there is nothing to record. But succeeding on the first attempt and succeeding on the third are different.

I need this number to know whether a prompt change helped. Looking only at the final result makes both look like success.

Rejection reasons accumulate in attempt order.

// SummaryEngine.swift:460-468
case .reject(let reason):
  rejections.append(reason.rawValue)   // 몇 번째 시도에서 왜 거부됐는지
  modelDraft = nil
}
} catch is CancellationError {
  throw CancellationError()
} catch {
  rejections.append(SummaryJudgment.Reason.modelFailed.rawValue)
  modelDraft = nil
}

Cancellation is different. Closing the screen is not a failure, so I do not record it and let it propagate unchanged.

The fallback ladder has four steps

The flow for making one chunk is this.

Order	What happens	If it fails
1	Ask the model	Record the reason and go to 2
2	Check whether it passes judgment, up to three times	If all three fail, go to 3
3	Extract sentences from the source	If there are none, go to 4
4	Put an empty chunk in place	—

At the second step I do not simply call it again. I include earlier rejection reasons in the next instruction. Three identical requests produce three identical answers.

The third step is extraction.

// SummaryEngine.swift:505-527
if let extracted = ExtractiveSummary.summarize(text: chunk) {
  let fields: [String]
  do {
    fields = try await localizeFields(...)
  } catch let error as ... {
    // 모델 초안과 달리 발췌는 원문에서 고른 값이므로 원문 언어 그대로 보존해도 안전하다.
    rejections.append(error.reason.rawValue)
    fields = [extracted.headline] + extracted.points
  }
  return SummaryPiece(index: index, ..., origin: .extracted, rejections: rejections)
}

Extraction does not call the model. It selects and arranges sentences from the source. It is not a summary, but the user can see what they sent.

Extraction has conditions under which it returns nothing

Extraction is not a verbatim copy. It selects and cuts sentences. If it cannot guarantee its own compression ratio, it returns nothing.

Value	What it prevents
Compression cap 0.4	Discards a result over 40% of the source as not a summary
Point length cap 120 characters	Above this, it reads like a paragraph rather than a sentence
Headline cap 60 characters	It must read as one line in a list
Minimum sentence 8 characters	Headers and page numbers fail here

The score for selecting a sentence is fixed too. A number gives points equal to its count (up to 6), % or percent gives 2, won, people, or items gives 1, a length between 20 and 120 characters gives 2, and one of the first three sentences gives 1. This scoring moves sentences containing measurements upward.

The title is not selected by score. If the first line is between 8 and 60 characters, I use the author's title. Choosing a title by score alone could make one table row or one KPI represent the whole record.

Repeated sentences are common in originals: a header repeated on every page or a scan error. I fold duplicates before choosing. Otherwise the extracted result inherits the repetition.

The last step is an empty chunk.

// SummaryEngine.swift:530-531
return SummaryPiece(
  index: index, headline: "", points: [], origin: .none, rejections: rejections)

The index remains even in an empty chunk. The order lets the screen say “The third part could not be summarized.”

Audio describes rejected ranges by time

Transcription has one more state. The recording is split into chunks for summarization, and one chunk is caught by the safety filter.

I mark that chunk excludedBySafetyPolicy and finish the other summaries. The result becomes readyWithExclusions rather than ready, and the screen says which minutes and seconds were excluded.

The whole point of this path is leaving the rejected chunk's place as a time range instead of deleting it. The user knows which part of their recording is missing and can listen to that range directly.

I do not translate extracted sentences

There is a separate judgment at the translation stage.

A model draft can be translated. If the model answers in the source language while the user's setting is English, I need to translate it.

Extraction is different. It is a sentence selected unchanged from the source. If translation fails, leaving it in the source language is safe because it is the text the user sent.

The comment records this distinction.

모델 초안과 달리 발췌는 원문에서 고른 값이므로 원문 언어 그대로 보존해도 안전하다.

If translation of a model draft fails, I run one more check, a gate that verifies the model-made sentence is grounded in the source. Extraction does not need that gate.

I also check the translator's result

I do not blindly trust the translator supplied by the app.

If I provide five sentences and four come back, the order is broken. I also check that the translated sentences are actually in the target language.

Check	If it fails
Is the count the same?	Discard the translation
Is it the target language?	Use the source text

Without both checks, I would not notice Korean coming back when I asked for Korean to be translated into English.

I propagate nine kinds of failure

The app's four branches become nine branches in the core.

// ProfileContracts.swift:387-397
public enum ProfileFailure: Codable, Hashable, Sendable {
  case invalidInput(String)                       // 입력이 비었거나 규격 밖
  case contextWouldOverflow(estimatedTokens: Int) // 창을 넘길 것으로 추정
  case malformedModelOutput(String)               // 모델이 규격을 벗어남
  case modelUnavailable                           // 모델을 쓸 수 없는 상태
  case modelFailed(String)                        // 호출 자체가 실패
  case modelCallBudgetExceeded(limit: Int)        // 요청당 호출 상한 초과
  case unsupportedProfile(ProfileIdentifier)      // 등록되지 않은 프로필
  case missingCheckpoint                          // 이어갈 기록이 없음
  case executionStoreUnavailable                  // 저장소를 열 수 없음
}

The first six concern the model and input. The last three concern adapters supplied by the app.

This distinction determines the screen text. For modelUnavailable I explain Settings; for modelCallBudgetExceeded I say the document is too long; for executionStoreUnavailable I tell the user to check storage.

app/AGENTS.md states the rule in one line.

모든 모델 단계에 폴백이 있어야 하고, 폴백은 언제나 "기록으로 남긴다"다.

Even if summarization fails, the user's text remains. Keeping the original is independent of the success of derived work.