Archiving and unarchiving structs, enums and other values to NSData and back - possible solution

Quincey, I'm going to mark your reply as the correct answer because I think you deserve the point the most.


I took your solution and made a working system, combining yours, mine and Monicker's ideas.

Although my solution suffers from the "multiplying number of cases problem" you outlined above, it is actually more suitable for what I needed to do (I needed to get rid of those switches in your deserialize function due bigger code architecture issues that may not apply in your case).


The way you used generics in your deserializedValueWithParameter function to bypass the "multiplying number of cases" problem was actually extremely beautiful and clever, but I wasn't able to find a solution that would allow me to get rid of the centralized information in your switches and keep your function, so it had to go in my case. But hats off to you for coming up with that solution, it may come in handy to other people with needs a bit different than mine.


The whole solution is below. I'll probably make a new thread with just 1 question and 1 answer so that other people can find this more easily. Thanks Quincey, Monicker and monyshuk. By the way if someone finds a way to make the code below better, please reply.


//: Playground - noun: a place where people can play

import UIKit

// MikeA, QuinceyMorris and Monicker contributed ideas to this code.


// The actual values represented by this type in the
// code below correspond to JSON. No fancy types are used.

typealias Archive = Dictionary<NSString, NSCoding>
protocol ArchivableValue {
    static var archiveTypeName: String { get }
    var archive: Archive { get }
    init (archive: Archive)
}

// This is a global register for archivable types.
// Each type capable of archiving must register itself
// by calling the registerType function, like so:
// registerType(Int)
// registerType(B<Int>)

var registeredTypes: [String: ArchivableValue.Type] = [:]
func registerType(t: ArchivableValue.Type) {
    registeredTypes[t.archiveTypeName] = t
}

// Functions for converting an ArchivableValue to Archive and back.

func archiveFromValue(value: ArchivableValue) -> Archive {
    var typedArchive: Archive = ["@type": value.dynamicType.archiveTypeName]

    for (key, value) in value.archive {
        typedArchive[key] = value
    }

    return typedArchive
}

func valueFromArchive(archive: Archive) -> ArchivableValue {
    let typeName = archive["@type"] as! String
    if let type = registeredTypes[typeName] {
        return type.init(archive: archive) // <- probably the most fancy part
    }
    fatalError("Unknown type \(typeName), please register it for unarchiving with registerType(\(typeName)) call")
}

// Convenience overload, this function makes the actual archiving/unarchiving code easier to read

func valueFromArchive(value: NSCoding?) -> ArchivableValue {
    return valueFromArchive(value as! Archive)
}

// Functions for converting Archive to NSData and back

func dataFromArchive(archive: Archive) -> NSData {
    return NSKeyedArchiver.archivedDataWithRootObject(archive)
}
func archiveFromData(data: NSData) -> Archive {
    return NSKeyedUnarchiver.unarchiveObjectWithData(data) as! Archive
}

// Convenience functions for converting ArchivableValue directly to NSData and back

func dataFromValue(value: ArchivableValue) -> NSData {
    return dataFromArchive(archiveFromValue(value))
}
func valueFromData(data: NSData) -> ArchivableValue {
    return valueFromArchive(archiveFromData(data))
}




// Below follows an actual implementation of the ArchivableValue protocol on several important types.
// Int, String, simple Struct, generic Struct and generic Enum are covered in the examples below.

// How to make existing value types archivable:

extension Int: ArchivableValue {
    static var archiveTypeName: String { return "Int" }

    var archive: Archive {
        return ["value": self]
    }

    init (archive: Archive) {
        self = archive["value"] as! Int
    }
}

extension String: ArchivableValue {
    static var archiveTypeName: String { return "String" }

    var archive: Archive {
        return ["value": self]
    }

    init(archive: Archive) {
        self = archive["value"] as! String
    }
}

// How to make custom struct archivable:

struct A: ArchivableValue {
    static var archiveTypeName: String { return "A" }

    var a1: Int
    var a2: String

    var archive: Archive {
        return [
            "a1": a1,
            "a2": a2
        ]
    }

    init(a1: Int, a2: String) {
        self.a1 = a1
        self.a2 = a2
    }

    init(archive: Archive) {
        a1 = archive["a1"] as! Int
        a2 = archive["a2"] as! String
    }
}

// How to make custom generic struct archivable:

struct B<T: ArchivableValue>: ArchivableValue {
    var b1: Int
    var b2: String
    var b3: T

    init (b1: Int, b2: String, b3: T) {
        self.b1 = b1
        self.b2 = b2
        self.b3 = b3
    }

    init (archive: Archive) {
        b1 = archive["b1"] as! Int
        b2 = archive["b2"] as! String

        // Types that do not conform to NSCoding protocol (nor are they bridged to
        // Cocoa class that conforms NSCoding protocol) must be encoded and decoded
        // using the archiveFromValue and valueFromArchive functions
        b3 = valueFromArchive(archive["b3"]) as! T
    }

    static var archiveTypeName: String {
        return "B<\(T.archiveTypeName)>"
    }

    var archive: Archive {
        return [
            "b1": b1,
            "b2": b2,

             // Types that do not conform to NSCoding protocol (nor are they bridged to
             // Cocoa class that conforms NSCoding protocol) must be encoded and decoded
             // using the archiveFromValue and valueFromArchive functions
            "b3": archiveFromValue(b3)
        ]
    }
}

// How to make custom generic enum archivable:

enum E<T1: ArchivableValue, T2: ArchivableValue>: ArchivableValue {

    case Option1
    case Option2(T1, T2)

    init (archive: Archive) {
        let tag = archive["tag"] as! String

        switch tag
        {
        case "Option1":
            self = Option1
        case "Option2":

            // Types that do not conform to NSCoding protocol (nor are they bridged to
            // Cocoa class that conforms NSCoding protocol) must be encoded and decoded
            // using the archiveFromValue and valueFromArchive functions
            let e1 = valueFromArchive(archive["e1"]) as! T1
            let e2 = valueFromArchive(archive["e2"]) as! T2
            self = Option2(e1, e2)
        default:
            fatalError("Could not decode enum E, invalid tag: \(tag)")
        }
    }

    static var archiveTypeName: String {
        return "E<\(T1.archiveTypeName),\(T2.archiveTypeName)>"
    }

    var archive: Archive {
        switch self
        {
        case .Option1:
            return [
                "tag": "Option1"
            ]
        case let .Option2(e1, e2):
            return [
                "tag": "Option2",

                // Types that do not conform to NSCoding protocol (nor are they bridged to
                // Cocoa class that conforms NSCoding protocol) must be encoded and decoded
                // using the archiveFromValue and valueFromArchive functions
                "e1": archiveFromValue(e1),
                "e2": archiveFromValue(e2)
            ]
        }
    }
}



// This is probably the weakest point of this archiving approach.
// Unfortunately, it is necessary to register each type that can be archived
// and unarchived using the registerType() call.
//
// The worst thing about this is that this may lead to combinatorial explosion
// when dealing with many different generic structs that may be type parameterized
// with many different other types.
//
// As soon as Swift provides a way to construct a generic Type (metatype?) value
// at runtime and pass it as a parameter to function, this can be simplified
// drastically. Maybe something like Type(type: Any.Type, parameters: [Any.Type]) -> Type  ?

registerType(Int)
registerType(String)
registerType(A)
registerType(B<A>)
registerType(B<Int>)
registerType(E<Int, String>)
registerType(B<String>)
registerType(E<Int, B<String>>)


// You can play with the whole system by uncommenting
// different values below. The startValue is converted
// into NSData, then, the NSData is converted back to
// the original value.


//let startValue = A(a1: 4, a2: "test")
//let startValue = B(b1: 4, b2: "test", b3: A(a1: 10, a2: "inner"))
//let startValue = B(b1: 4, b2: "test", b3: 123)
//let startValue = E<Int, String>.Option2(1, "test")


let startValue = E<Int, B<String>>.Option2(1, B(b1: 4, b2: "test", b3: "test2"))
let data: NSData = dataFromValue(startValue)
let endValue = valueFromData(data)

print(endValue)

Replies

Happy to see that you finally got what you wanted : ) even though I guess it's not as elegant and maintainable as you'd like it to be.


Our "conversation" in the main thread went the way it did because I couldn't / wouldn't believe that you would actually be satisfied with a solution that requires the code on the unarchiving side to spell out the complete type information of the (generic) struct to be unarchived, like eg E<Int, B<String>>.


I thought you were asking for something that would be able to decode the (generic) type information from the archive. That's how I read "deserializing generic structs".


The inevitable "lack of flexibility" of this solution (and my brain trying to interpret your question) could perhaps be made more clear at the end of your example code, like eg:


let startValue = E<Int, B<String>>.Option2(1, B(b1: 4, b2: "test", b3: "test2"))
let data: NSData = dataFromValue(startValue)
let endValue = valueFromData(data)
let usableEndValue = endValue as! E<Int, B<String>> // <-- Deserializing code has to know the exact type information, since it is of course impossible to get that from the archive.


You and others might think this is trivial and unnecessary, but judging from questions at SO and elsewhere, I believe a lot of people might actually expect something like this to work without the deserializing code having to specify the exact type information of whatever might be in the archive.



EDIT:

Sorry but I also just have to ask something regarding these comments in the code:


// As soon as Swift provides a way to construct a generic Type (metatype?) value
// at runtime and pass it as a parameter to function, this can be simplified
// drastically. Maybe something like Type(type: Any.Type, parameters: [Any.Type]) -> Type  ?


My naive reaction to this is:


So you want "runtime generic type construction", isn't that a contradiction in terms?

Moving generics away from compile time into runtime would AFAIK defeat the very purpose of generics.

Wouldn't this "runtime generics" just be a strange way of reinventing dynamic typing?


Someone please explain what it is that I don't understand, or give pointers to resources where I can learn more about these topics.

Maybe a better phrasing would be that I want "runtime parameterized type construction" instead of "runtime generic type construction". For example, I want to be able to construct Array<Int> or Array<String> type (not instance of that type, but the type itself) at runtime, in a convenient way.


Look at this dictionary and function:


var registeredTypes: [String: ArchivableValue.Type] = [:]
func registerType(t: ArchivableValue.Type) {
    registeredTypes[t.archiveTypeName] = t
}


The function has a "t" parameter of type "ArchivableValue.Type". This is similar to using type "Class" in Objective-C. You use it when you want to, for example, send a class itself (not an instance of the class) as a parameter to some method.


The dictionary variable "registeredTypes" has type [String: ArchivableValue.Type], which means it can give you a value representing atype that corresponds to a given String.


On line 44 in the example above, this happens:


return type.init(archive: archive) // <- probably the most fancy part


"type" is a value retrieved from the "registeredTypes" dictionary. Then, initializer is called on that Type, whatever its actual runtime value might be. This might be Int, String or whatever conforms to the ArchivableValue protocol.


All of this indicates that Swift can represent a particular type (or maybe a metatype is the better word?) as a value that can be stored and used at runtime.

However, it appears there is currently no way of creating/retrieving such type value, aside from requesting it from an object/struct/enum at runtime (using the object.dynamicType property). These are all runtime things, not subject to the compiler/typechecker limitations.


Essentially, Swift is missing an alternative to Objective-C's NSClassFromString() function. Something like:


func typeFromString(typeName: String) -> Any.Type { ... }


However, the code above works around this limitation using the "registeredTypes" approach.

Unfortunately, Swift also appears to be missing a function like this:


func genericTypeWithTypeParameters(typeName: String, typeParameters: [Any.Type]) -> Any.Type<...> { ... }


Ideally, I would like to do this:


let arrayType: Any.Type = genericTypeWithTypeParameters("Array", typeFromString("Int"))
let array = arrayType.init() // <--- This will give me an instance of empty Array<Int> at runtime


Such function could be used to construct parameterized types at runtime, which would make the above archiving code better for the reasons explained in the comments near line 225.



This should all (in principle) be perfectly possible. Swift clearly already can manipulate types as values and also call static methods on such type values, so the only thing missing are the above functions. Maybe they are already implemented and I just didn't find them; maybe they are private for now. There shouldn't be any deep reason why such function can't exist in Swift. I think it will come, sooner or later.

The type for Array<String> is [String].


Is it really so hard to crasp that generics in Swift are compiletime only? At runtime you have to use real types. How should the compiler resolve generics for runtime only data?

No, [String] is syntax sugar for Array<String>.


And I guess the type of the type Array<String> is the meta-type Array<String>.Type.

func printTypeOf<T>(_ : T.Type) { print(T); print(T.Type) }
printTypeOf(Array<String>) // Will print:
// Swift.Array<Swift.String>
// Swift.Array<Swift.String>.Type

Similarly, the type of the type Int is the meta-type Int.Type. So the type Int is an instance of Int.Type.

(Not 100% sure about any of this though).



As to whether Swift will/can support runtime construction of generic types or not, I'm still not sure what that would mean, except that using it would of course remove any compile time dependent benefits such as type checking and various opportunities for optimizations (perhaps there's some JIT-thingy that I am not aware of which means some optimizations could still be possible?).


Anyway, I've seen that eg C# has some runtime generics kind of thing going on which I fail to really understand (I have no experience in C#). Also, I'm having a hard time understanding how, in general, something like that would be anything else than a reformulated form of dynamic typing (which we already have, in Swift as well as in eg Objective-C, I mean Swift has reference types, and we can use eg NSClassFromString and NSObject in Swift, or implement some similar stuff with pure Swift classes, in short: It is already possible to have dynamically typed variables (eg Any in Swift), so why reinvent that?).

If I had a more complete understanding of Swift's generics (parametric polymorphism?) and generics in general, it would probably allow me to see all this differently, so if anyone can fill me in on this it would be appreciated.

(Based on my current understanding of these things, I wonder: If a feature like the one we are talking about here was available in Objective-C, would you then be able to construct a new C struct type from a string at runtime? I'm pretty sure that would be both impossible and a little absurd. But Swift's implementation of its form of generics might perhaps allow for something similar, I guess ... maybe ... but what would be the point of it all? Why not just use reference types (classes) and some existing way of managing the dynamic typing / binding and dispatch where your program requires such flexibility? Ie what would be the point of shoehorning "static things" (value types, static binding, generics) into acting as "dynamic things"? Why not just use the "dynamic things" we already have?)

And I guess the type of the type Array<String> is the meta-type Array<String>.Type.


This is correct.


As to whether Swift will/can support runtime construction of generic types or not,
I'm still not sure what that would mean, except that using it would of course remove
any compile time dependent benefits such as type checking and various opportunities
for optimizations


You wouldn't use this feature in regular code, only very, very rarely. One specific scenario when this feature is useful (or indeed, crucial) is the unarchiving mechanism. Being able to archive and unarchive complex data structures conveniently, no matter from what they are composed (structs, enums, classes) is very important.


I wonder: If a feature like the one we are talking about here was available
in Objective-C, would you then be able to construct a new C struct type from
a string at runtime? I'm pretty sure that would be both impossible and
a little absurd.


Just two points:


(1) You cannot have generic structs in Objective-C. Structs in Objective-C are just sequences of bytes, basically. As such, they are really easy do archive and unarchive. Structs in Objective-C also cannot conform to protocols, and you cannot declare an array of different kinds of structs. So, the problem of unarchiving an array of structs that all conform to some protocol but really are different types of structs that happens in Swift just doesn't occur in Objective-C. Structs in Objective-C are very limited and you can easily archive and unarchive them using + valueWithBytes:objCType: and

objCType
(+ cast) method of NSValue.

So, to sum up, due to structs being much more limited in Objective-C, the problems we're trying to solve for structs in Swift just don't occur in Objective-C.

You'd have to use classes for some of the scenarios for which structs are used in Swift, because you have no choice in the first place.

(2) Swift structs are not really structs in the traditional sense. There are more like immutable classes with value semantics under the hood. You can, for example, define a Node Struct:


struct Node
{
     var name: String
     var leftChild: Node
     var rightChild: Node
}


This struct can be used to represent tree data structures of arbitrary size. You can't do this in Objective-C without using pointers and reference semantics. With structs in Swift, you can do it without pointers and with value semantics. Here's what I mean:


var littleTree = Node(name: "Plus",
     leftChild: Node("123", []),
     rightChild: Node("456", [])
)


The "littleTree" value corresponds to a tree with 3 nodes:

     "Plus"
    /      \
   /        \
"123"      "456"


But, I can now do this:


littleTree.name = "Minus"


and even this:


littleTree.leftChild.name = "789"


And the new value of the "littleTree" variable will be:


     "Minus"
    /      \
   /        \
"789"      "456"



See how easy that was to do? I only had to reassign some properties in the structure, and it updated automatically.

But these are not classes with reference semantics - these are structs with value semantics, being used to represent trees

of arbritrary size that you can manipulate easily.


This behavior would be VERY difficult to achieve with immutable classes in Objective-C. You'd have to manually

reconstruct the whole tree if you changed something deep down in the structure. Swift does this automatically under the hood with

its struct and enum types.



This means that you can, in principle, use structs in Swift to represent any complex document, such as Pages document or Keynote document.

And Swift works behind the scenes to make these sort of things efficient.


Why would anyone represent their complex documents in this way? There are many benefits. Undo, a notoriously difficult thing for complex documents, becomes really trivial to implement - just swap the value representing the entire document for the old one! You can even have several parallel versions of your document, all of them browsable in real time. And when you deal with concurrency, representing your document only with value types solves tremendous amount of deep issues you otherwise encounter when you have mutable data structures in your model.


So, that's the rationale behind using structs and enums for everything. Not suitable for every app, but a HUGE blessing for some of us.


But, for all of this to be useful, we need a way to archive and unarchive these data structures to files for persistency.

Maybe I am misunderstanding you but I do not get how you come to the conclusion that generics are compiletime only.

I know in Java for example there is type erasure so you can not do something like

if(myObj instanceof List<String>) { /**/ }

because the type parameter <String> is absolutely not available at runtime but this is NOT the case in swift.


Take a look at the following:

struct MyGeneric<T> {
    let val : T
}

let a = MyGeneric(val: "hello")
let b = MyGeneric(val: 42)

func printType(arg: Any) {
    print(arg.dynamicType)
}

// prints the full parameterized type correctly:
printType(a) /* MyGeneric<Swift.String> */
printType(b) /* MyGeneric<Swift.Int> */



func isStringGeneric(gen : Any) -> Bool {
    return gen.dynamicType == MyGeneric<String>.self
}

// compared the the type correctly
print(isStringGeneric(a)) /* true */
print(isStringGeneric(b)) /* false */

func dynamicConstructor(gen : Any) -> Any? {
    if let t = gen.dynamicType as? MyGeneric<String>.Type {
        return t.init(val: "blub")
    } else {
        return nil
    }
}

// The full typevalue (including type parameters) can be retreived at runtime and be used to construct a new instance
print(dynamicConstructor(a)) /* Optional(MyGeneric<Swift.String>(val: "blub")) */
print(dynamicConstructor(b)) /* nil */


// Two parametrizations of the same generic type differ at runtime
func dynamicConstructorFail(gen : Any) -> Any {
    let t = gen.dynamicType as! MyGeneric<String>.Type
    return t.init(val: "foofoo")
}
print(dynamicConstructorFail(a)) /* MyGeneric<Swift.String>(val: "foofoo") */
print(dynamicConstructorFail(b)) /* signal SIGABRT because b's type (MyGeneric<Int>) can not be casted to MyGeneric<String> => the type parameter is not erased, MyGen<String> and MyGen<Int> differ at runtime  */


Clearly the full type information including the generic's types are available and useable at runtime.


PS (Not exactly to you Tia, but I dont want to create yet another answer for this):

Rethinking about the huge discussion in the other thread I am slowly guessing there is a misunderstanding of the difference between classes and structs in Swift. In (objective) C structs are just declarations of memory layout just used at compile time to calculate (pointer) offsets. When you come from this it is obvious that you think "oh you can not restore structs and genric types at runtime because they are just a compile type concept" but this is not the case in swift. As MikeA pointed out above, in Swift structs are mostly the same as classes except for their value semantics.

The same misunderstanding seems to exist regarding generics and the type system as a whole: Yes the type's are checked during compile time, but no they are NOT a PURE compile time construct because every value can be asked for it's dynamic type:

// in module A
public func erase(v: Any) -> Any {
    return v as Any
}

// in module B
printType(erase(a)) // still MyGeneric<Swift.String>


So when MikeA asks for creating an object of a dynamic but SPECIFIC type at runtime and you think that types are pure compile time thing then you naturally say "oh you can not do that! types are just for the compiler to check. They are good for nothing at runtime" but as you see above thats not quite right.

Well the longer of the 2 doesn't even work, throws an "Execution was interrupted, reason: signal SIGABRT." at line 41.

Yes because of the last line, which shows that MyGeneric<Int> and MyGeneric<String> (the types, not the instances) can not be cast into each other.

Remove the last line and it will work.

Thanks for that great explanation / motivation. I guess I might have a too "static view" on Swift's value types and generics, although I am (and was) very well aware of the fact that Swift's struct, class and generics are very different from let's say C++'s struct, class and templates. However I'm not sure I'm willing to accept everything you say about Swift's value types and generics.


First of all, your example isn't even a valid struct (I guess you might have intended it as just pseudo code, but the distinctions I'm about to make here are important to the discussion):

struct Node { // <--- COMPILE ERROR: Recursive value type 'Node' is not allowed
    var name: String
    var leftChild: Node
    var rightChild: Node
}


So, recursive value types aren't allowed, and why is that? Well, presumably because stuff like size, alignment, layout etc are important to the compiler (at compile time naturally), and so you'll need some sort of indirection (by reference) to get a thing like Node to work (just as you would need to in C or C++, and just as it would be "by default" using eg Objective-C (classes) or Javascript). So you'll need to do something like for example this:


struct Node {
    var name: String
    var leftChild: [Node] // Array struct that manages some data capable of holding zero or more child nodes.
    var rightChild: [Node] // Same here.
}


or this:

protocol NodeType {
    var name: String { get set }
    var leftChild: NodeType? { get set }
    var rightChild: NodeType? { get set }
}
struct Node : NodeType {
    var name: String
    var leftChild: NodeType?
    var rightChild: NodeType?
}


A third "workaround" would be to write a Box<T> class and use "Box<Node>?" for the children, which also means indirection by reference, and as we heard Chris Lattner say on WWDC you will (in a future beta) be able to use an "indirect" keyword to let the compiler do something similar to this behind the scenes, but it wouldn't change anything except making these things easier to write. You could of course also use a Swift class to make your Node type (with Node? for the children) thus perhaps making the necessary indirection/reference-thing even more clear.


But no matter how something like this is done, you will always have to introduces some indirection in it, and your concrete types (possibly of generic origin) will always have to be known to the compiler. Think about the fact that you will probably always be able to eg get a definitive answer when calling sizeof(Node). Well, I guess the size might of course depend on things such as target architecture and other compiler flags, but you know what I mean anyway.


I feel as though what you wrote didn't really change my understanding of Swift's value types and generics, what their possibilities and limitations probably are. That Node struct is a value type, yes, but it still has to use references/indirections internally, and the same is true for eg Array.


And as for generic type information being available at runtime, I would rather say that we can have information about a concretized generic type (as eg Array<Int> or Array<String>) that has been created by code that was generated by the compiler at compile time using the necessary parts for that, namely:

1. Some "generic type itself" (eg Array<T>, note the T, unspecifed) and

2. Some other types to take the place of eg that T.

and this doesn't necessarily mean (AFAICS) that we will (in the future) be able to construct / concretize generic types at runtime (from for example a string representation, by interpreting the string as giving us the necessary parts, ie point 1 and 2 above).


Another way of saying approximately the same thing is that I see no reason for why we would ever be able to do something like eg:

let t: Any.Type = Array // This will not compile (probably not even in some future Swift version).

Array by itself (without anything for its T) is only meaningful for the compiler. And the compiler will not be satisified with code such as above unless you specify what its T should be. And I doubt that we will ever be able to do something like this in Swift.


IMHO it might be like this: At runtime, it certainly is possible to check that something is eg an Array<Int> or an Array<String> (if the compiler has generated the code that created the value of that type), but you can't check for or create something like the Array<T> (note the T), because Array<T> is the "unconcretized generic type itself" which is something that is only available to the compiler, for the sole purpose of making it possible for it to create concretized types such as Array<Int> or Array<String> from it.


I think that if there is some uncertainty at compile time wheter a value of some unconcretized generic type, eg Array, will be eg Array<String> or Array<Int>, then you will only be able to construct eg Array<Any>, and you will not be able to construct an Array<Int> or Array<String>, because the compiler can not know which it will be, so it will not be able to generate the code to do it. Simple as that. (But then again, I'm not sure about any of this and I'm trying to learn more about it. But it isn't exactly easy to find information about it.)


It's easier to explain what I mean using some code:

import Darwin // (Just for arc4random)

func getSomeValueOfATypeWhichIsKnownOnlyAtRuntime() -> Any {
    if arc4random() > UInt32.max / 2 { return 123 }
    else { return "Hello world" }
}
struct Foo<T> {
    let value: T
    init(_ value: T) { self.value = value }
}
func test() {
    let a = Foo(12345); print(a) // Foo<Int>, of course.
    let b = Foo("Abc"); print(b) // Foo<String>, of course.

    // This will of course also work:
    let c = Foo(getSomeValueOfATypeWhichIsKnownOnlyAtRuntime())

    // BUT: It will always result in c being Foo<Any>, and not Foo<Int> or Foo<String>:
    switch c {
    case let c as Foo<Int>: print(c)    // Warning: Cast from 'Foo<Any>' to unrelated type 'Foo<Int>' always fails
    case let c as Foo<String>: print(c) // Warning: Cast from 'Foo<Any>' to unrelated type 'Foo<String>' always fails
    default: print(c)
    }
    // The switch will always print Foo<protocol<>>(value: ***)
    // That is, at runtime, c is always Foo<Any> and you'd have to
    // force cast it into either Foo<String> and Foo<Int>,
    // knowing what it should be.
}
test()


The following might seem to contradict the above, but the comments are there to explain the difference (ie the compiler can know the constituent parts of the concretized generic type when it generates the code that creates the value of that type, it doesn't matter that the compiler can't know which branch of the if statement will be taken, it is still the compiler that generates the code in each branch):

import Darwin
struct Foo<T> {
    let value: T
    init(_ value: T) { self.value = value }
}
let a: Any
if arc4random() > UInt32.max {
    a = Foo(12345) // Compiler can generate code here for constructing a Foo<Int>
} else {
    a = Foo("Abc") // Compiler can generate code here for constructing a Foo<String>
}
switch a {
case let a as Foo<Int>: print("Int!")
case let a as Foo<String>: print("String!")
default: print("I'm not going to get printed.")
}


Again, I'm more than willing to accept that my views on this are totally wrong. The only reason I keep nagging about this is that I really want to learn Swift, and as detailed information about this particular topic is hard (impossible?) to find, all I can do is really to experiment / ask / discuss / argue. : )

For arguments sake, I will say that your example isn't really doing anything with generic types that is only knowable at runtime.

Please see the reply I just wrote to MikeA above, here:

https://forums.developer.apple.com/message/21041#21041


See especially my last two code examples, at the bottom of the post. They show that if it's only knowable at runtime what the X should be for a Foo<X>, and X might be Int or String, then you can't really create a value of type Foo<Int> or Foo<String>. No matter what you do, you'll only be able to create a Foo<Any> which you'll then have to force cast into being eg Foo<Int> or Foo<String>. You'll have to know in some other way which of them it should be because the runtime will not provide that knowledge.


However, if you write code for the construction of each possible specific type (eg Foo(123) and Foo("Hello"), or just the type: Foo<Int>, Foo<String>) and then leave the choice about which of those sections of code should be run to be decided at runtime, then that's not at all the same thing. The compiler has generated the relevant code already, it's just which parts of that code will be run that is up to the runtime ... Anyway, all this is expressed a lot clearer in the above answer.

I originally had the Node example written something like this:


struct Node {
     var name: String
     var subnodes: [Node]
}


But I changed this later to leftNode and rightNode to make the example more simple. I didn't realize it won't actually compile till Apple implements the "indirect" modifier.


Anyway, this doesn't affect the rest of the things I wrote concerning the advantages and behavior of value types, and why that behavior (referential transparency, immutability and copy-on-write semantics) is desirable.


So, recursive value types aren't allowed, and why is that? Well, presumably
because stuff like size, alignment, layout etc are important to the compiler (at compile
time naturally), and so you'll need some sort of indirection (by reference) to get a thing
like Node to work (just as you would need to in C or C++, and just as it would be
"by default" using eg Objective-C (classes) or Javascript). So you'll need to do something
like for example this:


I think you are slightly wrong here. Yes, some indirection is needed, but the Swift structs are unique in that they will still preserve the value semantics. This is totally different to how it would behave if you simply used the equivalent of the above Node struct in Objective-C or C++. If you choose to represent a whole tree using the Node structure in Swift, the whole thing will behave as a single value. You'll be able to do:


var node = ... create whole tree ...
node.subnodes[0].subnodes[1] = someOtherNode
... node itself is a different value here ...


Of course, under the hood, this is probably implemented via dynamic data structures and pointers, but towards the programmer, it continues to behave as if the entire node and the arbitrary large tree it represents were a single value.


By modifying a deep, nested node, you've also modified all its parent nodes. More precisely, copies of your original structs were made in memory under the hood to maintain the appearance of value semantics. This is what "copy-on-write" semantics means - when you make a change to your struct, even if the change is nested deeply in chain of several structs, a copy of the necessary portion of the structs is made and a new value is assigned to your "node" variable to maintain the illusion of value semantics, even if all of this is implemented via indirection and references under the hood. (At least that is my guess about how this is all implemented. I'm sure the compiler can optimize this even further in some cases).


I'm doing a terrible job explaining this, but the bottom line is, even deeply nested structs with arbritrary complexity will behave as a single value in Swift. This is an extremely unusual language feature, but it might be very useful due to reasons I described in previous post.


As I said before, it is better to think about structs in Swift as if they were immutable classes with value semantics, automagically replaced when you attempt to mutate them with a new, immutable value. There is very little technical similarity with classic C structs.



Regarding the second part of your reply, you're doing something subtly different in your examples. You are using generics to make the new value on lines 9 and 10. Generics, the way you used them here, are indeed a fully compile time constructs, and the same approach cannot be used to construct a value of arbitrary type at runtime.


Instead, consider this:



protocol ArchivableValue {
    init ()
}

struct A: ArchivableValue {   
}

struct B: ArchivableValue {
}

struct C<X>: ArchivableValue {   
}

func createValueOfType(type: ArchivableValue.Type) -> ArchivableValue {
    return type.init()
}

var type: ArchivableValue.Type = A.self
let x = createValueOfType(type)

type = B.self
let y = createValueOfType(type)

type = C<Int>.self
let z = createValueOfType(type)

print(x) // "A()"
print(y) // "B()"
print(z) // "C<Swift.Int>()"


The createValueOfType function takes a type (of metatype ArchivableValue.Type) and returns a new value of that type you specified.


The example does tihs on lines 19, 22 and 25. Type itself is sent as a parameter to the createValueOfType() calls.

And new values of the correct type are returned from that function call.


Crucially, the createValueOfType function is not generic. It doesn't rely on generics or any compile-time thing to do what it does.


To convince yourself of this, you can append this code at the end of the above example:


let randomBool = arc4random_uniform(2) == 0
let w = createValueOfType(randomBool ? x.dynamicType : y.dynamicType)
print(w)


Here, I use a random number to determine which type will be used for the creation of the "w" value.


Clearly, the actual value of "w" and the actual type of the value stored in "w" is determined at runtime (the compiler can only know that "w" is ArchivableValue). This demonstrates that the createValueOfType function is fully runtime-parameterized with the type parameter, and also demonstrates that Swift can represent types themselves, even at runtime. Moreover, it can use this types as if they were values and call initializers on them (see line 15 in the second-last example).

We are essentially in agreement about the struct-part. I just wanted to point out that it's still indirection, be it up-front or behind the scenes, and Swift structs are only preserving value semantics as long as we make them do that (by eg leaving it up to the Array holding our subnodes or (presumably) by using the soon to come indirect modifier). It is totally possible to make Swift structs that does not preserve value semantics. So for example:


This Node struct will preserve value semantics (by leaving the heavy lifting to the impl of Array):

struct Node {
    var name: String
    var subnodes: [Node]
}
var tree = Node(name: "Root", subnodes: [
    Node(name: "Foo", subnodes: []),
    Node(name: "Bar", subnodes: [])
    ])
var treeCopy = tree
tree.subnodes[0].name = "Baz"
print(treeCopy.subnodes[0].name) // Foo (thanks to Array's implementation which includes copy on write)
print(treeCopy.subnodes[1].name) // Bar


While this Node struct will not preserve value semantics (because our naive optional Box strategy won't make that happen):

final class Box<T> {
    var value: T
    init(_ value: T) { self.value = value }
}
struct Node {
    var name: String
    var leftChild: Box<Node>?
    var rightChild: Box<Node>?
}
var tree = Node(name: "Root",
    leftChild:  Box(Node(name: "Foo", leftChild: nil, rightChild: nil)),
    rightChild: Box(Node(name: "Bar", leftChild: nil, rightChild: nil))
)
var treeCopy = tree
tree.leftChild?.value.name = "Baz"
print(treeCopy.leftChild?.value.name) // Baz, not Foo : ( (because the naive Box strategy will not allow this struct to preserve value semantics)
print(treeCopy.rightChild?.value.name) // Bar


So, all I'm saying is: I do understand everything you say and I agree that Swift's structs are great. But IMHO their "value semantics preserving powers" are not *that* (auto)magical, of course they might appear to be more so when we have the indirect keyword but I see that as more of a cosmetic thing. It's still important for programmers to know how these things actually work, or they might easily be confused. But anyway, I think we can leave this question behind.



About the runtime generic/parametric type construction.


In order to see if there really are any differences of opinion, I think we can reduce the discussion to the following.


The way you are using this:

func createValueOfType(type: ArchivableValue.Type) -> ArchivableValue {
    return type.init()
}

is only by feeding it types that you have spelled out in your code, ie they are already "concretized" generic types, types with generic origins, and the compiler has of course generated the code for creating those types. So this belongs to the same category as the second of the two examples at the end of my previous reply.


But, the way I see it, this is not the same as what you were asking/waiting for here:

// As soon as Swift provides a way to construct a generic Type (metatype?) value
// at runtime and pass it as a parameter to function, this can be simplified
// drastically. Maybe something like Type(type: Any.Type, parameters: [Any.Type]) -> Type  ?

Correct me if I'm wrong, but what you were asking for here belongs to the same category as the first of the two examples (in my previous reply).


But let's not argue about if that is really the case or not. Not yet anyway.


Instead, let me just ask you if by this:

// ... something like Type(type: Any.Type, parameters: [Any.Type]) -> Type


you mean that you would like to be able to do something like this:

let gt: Any.Type = Array // gt is some "unconcretized generic type", could be Array, could be Optional, knowable only at runtime
let et: Any.Type = Int // et is some type that should take the place of gt's T, could be eg Int, also knowable only at runtime.
let t = Type(type: gt, parameters: [et]) // t is "concretized" at runtime from gt and et, so here t would now be usable as Array<Int>

?

you mean that you would like to be able to do something like this:

let gt: Any.Type = Array // gt is some "unconcretized generic type", could be Array, could be Optional, knowable only at runtime
let et: Any.Type = Int // et is some type that should take the place of gt's T, could be eg Int, also knowable only at runtime.
let t = Type(type: gt, parameters: [et]) // t is "concretized" at runtime from gt and et, so here t would now be usable as Array<Int>

?


Yes, exactly. And after that, I need to be able to cast "t" to at least some kind of protocol-conforming metatype variable:

let typedT: MyProtocol.Type = t as! MyProtocol.Type


// Do something with typedT that I can do with types of MyProtocol.Type metatype...
// such as typedT.init(fromArchive: archive)

I'm not sure if I can do this now (probably not), but both the things are necessary for the painless unarchiving process.

Ok, so finally : ) this is where our opinons differ, I think this will never be possible.


My view on this can be simplified even further:


I think we will never be able to, at runtime, handle/access any "unconcretized" generic type like 'Array' (that is: just 'Array', with its 'T' unspecified).


The reason is that I assume these "unconcretized" generic types are pure compile time constructs. And I think they will remain so even in future versions of Swift. No code examples in this thread have indicated anything else. The fact that an Any variable can hold a "concretized type with generic origins" like eg Array<String> and be asked for its dynamicType (resulting in printing the string "Swift.Array<Swift.String>") isn't really telling us anything, it's still the compiler that has had to compose that concretized type (using the unconcretized generic type 'Array' and the type 'String' as its type parameter). This is exactly what I meant to demonstrate by the subtle difference of the two code examples at the end of my earlier reply. The first one is trying to do something like what we are talking about here (the impossible), the other one is doing something that might seem to be the same thing, but if you think carefully and read the comments in the code, you'll see that it's actually not at all the same thing.


So, in short:

// I think something like this will never be possible, because
// without a <***>, 'Array' is a pure compile time concept:
let someType: Any.Type = Array


(And, if its <***> is specified, then at runtime it'll just be like any other type, the only difference being that it happens to have generic origins, which is something that the runtime is unaware of (yes I know about dynamicType, and that it might result in something like eg "Swift.Array<Swift.String>" being printed but that is really just a string/description that the compiler has generated for that type when it composed/concretized it, talking of which we should of course look and see what can be found out through the Mirror of eg an Array<Int>, using reflection, I haven't done that).)


Btw, as of Swift 2 beta 2, we can't even typealias Array without specifying its T:

typealias MyType = Array // Error: Reference to generic type 'Array' requires arguments in <...>


As usual, please correct me (in as much detail as possible) if I'm wrong about any of this.

I sort of agree. I would also be happy with this solution:


let gtName: String = "Array" // gt is some "unconcretized generic type", could be Array, could be Optional, knowable only at runtime
let et: Any.Type = Int // et is some type that should take the place of gt's T, could be eg Int, also knowable only at runtime.
let t = makeType(gtName, parameters: [et]) // t is "concretized" at runtime from gt and et, so here t would now be usable as Array<Int>


I.e. I don't insist on the first parameter being a Any.Type, using a String for the first parameter of the makeType function would also work for the unarchiving case I care about.


That being said, Apple still might let us use "Array.self" as an expression some day.

It would have to return something other than a full, realized type (as you correctly pointed out, "Array" by itself isn't a valid type), but it may still return

some useful thing that can be, for example, used to construct the fully specified type.